Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-08-27
15:51:53 sean-k-mooney then it should work
15:53:39 dansmith sean-k-mooney: why will it fail without kvm? is it because the host will have no numa info so we'll fail to schedule anything?
15:54:37 sean-k-mooney no it fails becasue when we set hw:numa_node=1 we pin the guest cpus to float over just 1 host numa node
15:54:59 sean-k-mooney we do that using the vcpupin element instad of the cpuset attibute on the vcpu element
15:55:22 sean-k-mooney the vcpupin element which does per core pinning is only supported with kvm or the mttcg backend
15:55:51 dansmith ah okay
15:56:17 sean-k-mooney the normal tsg backend that qemu ueses only support pinning via the <vcpu cpuset=""> method
15:56:45 sean-k-mooney so in pricaipal there is no reason it cant work just the way we generate the xml breaks it
15:57:13 sean-k-mooney or use a newer qemu/libvirt
15:57:33 sean-k-mooney i belive the one in fedroa 31 will be new enough by defualt
15:57:50 sean-k-mooney f29 and i think f30 still need the virt preview repo enabeld
15:58:17 dansmith sean-k-mooney: maybe you could fix up that patch for artom to do whatever it is that needs doing?
15:59:19 sean-k-mooney i can try yes
15:59:25 sean-k-mooney i also need to fix https://review.opendev.org/#/c/652197/
15:59:33 sean-k-mooney that is my upstream nfv job
16:00:05 sean-k-mooney it uses the really new qemu form the virt preview repo but devstack/zuul jobs move a file or change the permisions
16:00:11 sean-k-mooney so its currenly broken
16:00:43 sean-k-mooney you can see i was testing hugepage + numa + pinning in the gate
16:00:45 sean-k-mooney https://review.opendev.org/#/c/652197/18/playbooks/nfv/nfv.yaml
16:01:51 sean-k-mooney if i can get that working again we can test artoms live migration stuff in the gate and stephenfin cpu pinning spec
16:03:23 sean-k-mooney The destination directory (/opt/stack/devstack) is not writable by the current user. Error was: [Errno 13] Permission denied
16:03:38 sean-k-mooney artom: is it /opt/stack/new/devstack?
16:05:17 artom sean-k-mooney, at this point, whatever's quicker/more demonstrative
16:05:50 sean-k-mooney i think if i jsut add "become: yes" to my patch it might work
16:05:51 openstackgerrit Stephen Finucane proposed openstack/nova master: scheduler: Flatten 'ResourceRequest.from_extra_specs', 'from_image_props' https://review.opendev.org/674894
16:05:51 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove 'hw:cpu_policy', 'hw:mem_page_size' extra specs from API samples https://review.opendev.org/675338
16:05:52 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Start reporting PCPU inventory to placement https://review.opendev.org/671793
16:05:52 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available' https://review.opendev.org/672693
16:05:53 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Rewrap definitions of 'NUMACell' https://review.opendev.org/674395
16:05:53 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800
16:05:54 openstackgerrit Stephen Finucane proposed openstack/nova master: objects: Rename 'fields' import to 'obj_fields' https://review.opendev.org/674103
16:05:54 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait https://review.opendev.org/675571
16:05:55 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Simplify 'fakelibvirt.HostInfo' object https://review.opendev.org/678861
16:05:55 openstackgerrit Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801
16:05:56 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Fold in argument to '_update_provider_tree_for_vgpu' https://review.opendev.org/676729
16:05:56 openstackgerrit Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895
16:05:57 sean-k-mooney ill do that and then look at yours
16:06:14 openstackgerrit Artom Lifshitz proposed openstack/nova master: DNM: Run LM integration tests with NUMA flavor https://review.opendev.org/678887
16:08:14 openstackgerrit sean mooney proposed openstack/nova master: Libvirt: report storage bus traits https://review.opendev.org/666914
16:08:15 openstackgerrit sean mooney proposed openstack/nova master: libvirt: use domain capabilities to get supported device models https://review.opendev.org/666915
16:08:15 openstackgerrit sean mooney proposed openstack/nova master: Add transform_image_metadata request filter https://review.opendev.org/665775
16:09:54 mriedem sean-k-mooney: we can override the virt_type in local.conf for the job to use kvm
16:10:04 mriedem since this is a hack test patch anyway
16:10:07 sean-k-mooney yes we can
16:11:24 sean-k-mooney artom: i think the nodeset you want is ubuntu-bionic-vexxhost but i need to double check
16:11:44 artom sean-k-mooney, yeah, I'm going in completely blind and relying on the gate to tell me what's wrong
16:11:50 artom So that's entirely possible
16:11:54 openstackgerrit Matt Riedemann proposed openstack/nova master: DNM: Run LM integration tests with NUMA flavor https://review.opendev.org/678887
16:11:57 sean-k-mooney you use it like this
16:11:59 sean-k-mooney https://opendev.org/openstack/magnum/src/branch/master/.zuul.yaml#L152-L158
16:12:24 artom mriedem, *tips fedora* your help is appreciated
16:13:33 sean-k-mooney do you want to test master our your change?
16:17:07 openstackgerrit sean mooney proposed openstack/nova master: Libvirt: add nfv job https://review.opendev.org/652197
16:17:43 sean-k-mooney i think ^ will fix my nfv job
16:19:16 sean-k-mooney the multi node version of that enable live migration alther i might need to add the flag to allow that
16:25:02 openstackgerrit sean mooney proposed openstack/nova master: Libvirt: add nfv job https://review.opendev.org/652197
16:36:10 openstackgerrit Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801
16:36:11 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Fold in argument to '_update_provider_tree_for_vgpu' https://review.opendev.org/676729
16:36:11 openstackgerrit Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895
16:36:12 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Start checking compute usage in functional tests https://review.opendev.org/678902
16:37:03 stephenfin bauzas: fwiw, I think I've everything in https://review.opendev.org/674894 resolved if you wanted to take another look
16:37:19 stephenfin I, however, am out of here o/
16:39:28 openstackgerrit sean mooney proposed openstack/nova master: DNM: Run LM integration tests with NUMA flavor https://review.opendev.org/678887
16:39:34 sean-k-mooney artom: mriedem dansmith i think ^ will fix it
16:45:46 efried stephenfin: you still around?
16:46:04 efried I've been in meetings for 2h wanting to talk to you about this, but afraid I may have missed my window :(
16:50:14 artom sean-k-mooney, ack, much thanks!
16:50:49 sean-k-mooney we should have 3 jobs trying to test numa with the gate as we speak
16:51:04 sean-k-mooney the singel and multinode jobs i wrote and running and yours
16:51:30 sean-k-mooney i assume i should be able to go test the latest version of your code again manually?
16:56:15 sean-k-mooney oh i just rediscovered somthing quite useful
16:57:11 sean-k-mooney the experimenatl queue is way fater the check. which i kind of knew other pipleine are faster but i forgot
16:57:34 openstackgerrit Matt Riedemann proposed openstack/nova master: Specify availability_zone to unshelve https://review.opendev.org/663851
17:03:07 mriedem alex_xu: for your tomorrow, please take another pass on ^
17:03:14 mriedem so we can get 2.77 in and move on
17:06:06 dansmith artom: it might be easier to discuss https://review.opendev.org/#/c/635669/39/nova/compute/resource_tracker.py here
17:06:36 artom dansmith, shoot :)
17:07:04 dansmith artom: my point is that you're assuming that if we're live migrating an instance, you can pass an empty PCIRequests object into the migration claim, because a live-migrating instance couldn't possibly have pci devices, right?
17:07:49 artom dansmith, no, because we don't want the claim to think it has any PCI devices
17:08:02 dansmith um
17:08:10 dansmith artom: why is that?
17:08:37 artom Because for a subset of PCI devices (SRIOV Neutron ports), that's handled buy Sean's code
17:09:00 artom So we don't want to refuse based something that should otherwise work
17:09:15 artom The missing piece here is that all other PCI devices *should* get refused
17:09:28 dansmith should get refused somewhere earlier in the stack right?
17:09:32 artom Yeah
17:09:49 dansmith that's the part I don't like
17:10:03 artom I agree - it's also current/latent
17:10:30 artom IIRC we just blow up if we try to migrate an instance with PCI alias-based passthrough
17:11:00 sean-k-mooney we check and fail the migration here https://github.com/openstack/nova/blob/master/nova/conductor/tasks/live_migrate.py#L216-L221
17:11:01 dansmith artom: it's not latent because it's the code you're adding
17:11:20 sean-k-mooney if the instance has a pcirequest that is not a neutron port we never call artoms code
17:11:51 dansmith sean-k-mooney: currently.
17:11:54 artom sean-k-mooney, aha, thanks for that - that confirms my idea that we should do claims *after* your MigrationPreCheck
17:12:10 artom So I'll have to change that
17:12:34 sean-k-mooney artom: https://github.com/openstack/nova/blob/master/nova/conductor/tasks/live_migrate.py#L317-L326
17:12:55 sean-k-mooney we do the pci check before calling check_can_live_migrate_destiation
17:13:11 sean-k-mooney so we should do the claims in check_can_live_migrate_destination
17:13:20 artom Ah, sorry, missed the wider context in your link. OK, then we're good.
17:13:22 dansmith artom: so my concern is that you say "we should never get here" and throw up an empty PCIRequests object, which I think goes into the migration context, and eventually gets applied to the instance

Earlier   Later