Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-09-18
16:31:36 sean-k-mooney mnaser: ya i dont know either. this is where i delegate to danpb or kashyap as i read C but dont really want to have to find where that happens
16:31:49 sean-k-mooney mnaser: its not a virsh issue
16:31:59 mnaser ok fair nuff
16:32:12 sean-k-mooney wehn we create teh xml and define the domain libvirt parses it and save an updated version to disk and in memory
16:32:49 sean-k-mooney virsh talks to libvirt over a unix socket and virsh dumpxml give you the rendered version
16:33:11 mnaser oooh
16:33:13 sean-k-mooney libvirt when it ingest the xml fills in a bunch of things like guest pci address that we dont set
16:33:15 mnaser wait so i can do cat on the local disk then
16:33:43 sean-k-mooney there is a copy saved to disk somewhere but you could also just check the nova compute log
16:33:47 sean-k-mooney the xml is saved in it
16:33:55 mnaser yeah not running $world with DEBUG on
16:33:59 mnaser so that'd have to be a little exercise :p
16:34:12 mnaser its only logged when running under debug afaik
16:34:21 sean-k-mooney yes it is
16:34:40 mnaser time to google for the millionth time "how to run an openstack instance on a specific node"
16:34:51 sean-k-mooney anyway i would check the qemu instance log and see what the error is
16:35:00 openstackgerrit Merged openstack/nova-specs master: Update spec: filtering of alloc candidates by forbidden aggregates https://review.opendev.org/675384
16:35:15 sean-k-mooney cat /var/log/libvirt/qemu/instanace...
16:35:33 mnaser yeah but i tihnk its a layer before that because the qemu process doesnt have the flag
16:35:34 mnaser "-cpu IvyBridge-IBRS,ss=on,pcid=on,hypervisor=on,arat=on,tsc_adjust=on,md-clear=on,stibp=on,ssbd=on,xsaveopt=on,pdpe1gb=on"
16:35:54 sean-k-mooney which flag is missing?
16:36:04 sean-k-mooney i though the vm was crashing?
16:36:06 mnaser vmx (and the fact that things are manually added)
16:36:13 mnaser nope, no crashes at all, just literally a flag missing
16:36:23 mnaser virsh capabilities shows vmx
16:36:32 mnaser but booting instances without defining cpu_model does not
16:36:40 sean-k-mooney right but did you turn nested virt on in the host kernel
16:36:42 mnaser does not boot it with vmx then
16:36:46 mnaser yep
16:37:05 sean-k-mooney ok i normally use host-passthough with nested virt so the flag is there
16:37:08 mnaser `virsh capabilities` wouldn't report vmx there if it wasnt on either afaik
16:37:11 mnaser yeah, im trying to avoid that one
16:37:17 sean-k-mooney the model your using might just not included it
16:37:21 sean-k-mooney you could jsut add it
16:37:26 mnaser cat /sys/module/kvm_intel/parameters/nested => Y
16:37:29 sean-k-mooney with the config
16:37:53 mnaser sean-k-mooney: it actually does though -- http://paste.openstack.org/show/777431/
16:37:54 mnaser thats the thing
16:38:09 mnaser i could add it manually but im trying to see if its something we can improve upstream :>
16:38:10 sean-k-mooney the host does
16:38:29 mnaser isnt the output of virsh capabiltiies show the model used by libvirt?
16:38:35 mnaser in this case IvyBridge-IBRS which does include vmx
16:38:36 sean-k-mooney that does not mean the IvyBridge-IBRS has the flag
16:38:43 mnaser wait what
16:38:45 mnaser seriously
16:38:51 sean-k-mooney yes
16:39:07 sean-k-mooney there is an xml file you can check
16:39:10 mnaser i thought virsh capabilities was what would host-model be
16:39:17 sean-k-mooney no
16:39:20 mnaser cpu_map.xml
16:39:28 sean-k-mooney its litrally the host capablites unfiltered
16:39:57 sean-k-mooney the model it reporet is the closet on to your host cpu
16:40:04 mnaser well
16:40:07 mnaser that explains all my confusion
16:40:14 mnaser then i need to add the extra cpu flag
16:40:28 mnaser cpu_map.xml shows nothing actually has vmx flag
16:40:37 mnaser i could _swear_ that it used to be there before
16:40:51 sean-k-mooney where is cpu_map by the way i was looking for it locally
16:41:03 mnaser sean-k-mooney: /usr/share/libvirt/cpu_map.xml
16:41:08 mnaser (on centos at least)
16:41:14 sean-k-mooney ah user share i was looking in /var/lib
16:41:54 mnaser ok im really confused as to how it used to work before but i guess it did
16:42:20 sean-k-mooney it could be a libvirt change
16:42:53 sean-k-mooney or you were using host-passthough and did not no/notice/remember
16:43:20 mnaser no as much as host-passthrough is great im very anti that because live migrations
16:43:51 sean-k-mooney you can use it with live migrations
16:43:59 sean-k-mooney but you need to group things by hardware type
16:44:08 mnaser yeah it gets tricky for new hardware
16:44:09 sean-k-mooney by the way host-model is not relaly better
16:44:40 sean-k-mooney if you use host model and you move for ivybridge to haswell then reboot the vm it wont be abel to migrte back
16:44:51 mnaser yeah im perfectly ok with migrating from old to new but not new to old
16:44:58 sean-k-mooney if you really care about migration you should use custom and select a version across all nodes
16:44:58 mnaser it enables my use case of "upgrading things"
16:52:26 stephenfin mriedem: Can you cut an sqlalchemy-migrate release?
16:53:17 stephenfin I'm seeing logs about a contextual_connect popping up and I assume that's not helping with our "we generate too many logs" gate issues. A patch from zzzeek is on master
16:55:33 sean-k-mooney stephenfin: we cant in train
16:55:48 sean-k-mooney but we can for U
16:56:01 mriedem stephenfin: i've got a release pending for U
16:56:08 sean-k-mooney stephenfin: i think mriedem may have fixed that
16:56:11 mriedem https://review.opendev.org/#/c/682656/
16:57:32 mriedem i see what you mean in https://7adee8d979f9b27778af-fc266c4961a026b4ec86218d0c17f3b6.ssl.cf5.rackcdn.com/523559/10/check/openstack-tox-py36/55f9ad4/job-output.txt and https://ce11c0ee0a74ee8ebb7e-9c035ca54e1e355b36ad4d338836f375.ssl.cf2.rackcdn.com/523559/10/check/nova-tox-functional-py36/ac3beea/job-output.txt though
16:57:42 mriedem stephenfin: in nova we can add a warnings filter to only log that once
16:57:49 mriedem in the WarningsFixture we have
16:57:51 sean-k-mooney stephenfin: speaking of that im updating there tox file here https://review.opendev.org/#/c/682515/ if you care too take a look. the native coverage support still seam to not work. i spend an hour working on it so i did the same hack we have in nova
16:58:09 mriedem stephenfin: so let's do that in nova in Train and if it's fixed in sqla-migrate we can bump mins when that gets released in U
16:58:40 mriedem you could push the nova patch under bug 1813147
16:58:41 openstack bug 1813147 in OpenStack Compute (nova) "p35 jobs are failing with subunit.parser ... FAILED" [High,In progress] https://launchpad.net/bugs/1813147 - Assigned to Balazs Gibizer (balazs-gibizer)
16:59:53 mriedem this is the patch you were talking about btw https://review.opendev.org/#/c/671040/
17:01:46 openstackgerrit melanie witt proposed openstack/nova master: Add note about needing noVNC >= v1.1.0 with using ESX https://review.opendev.org/682946
17:11:23 melwitt KeithMnemonic1: ^ (late)
17:15:39 KeithMnemonic1 lol
17:18:53 sean-k-mooney im not sure why i try to make my tests robost...
17:20:24 sean-k-mooney gibi: mriedem http://paste.openstack.org/show/777439/
17:20:48 sean-k-mooney so you know why stat is not ment to be HARD_REBOOT ...
17:22:50 sean-k-mooney im going to add backin self._wait_for_state_change(self.api, shelved_server, 'ACTIVE')
17:23:46 sean-k-mooney or active_server in this case
17:24:20 mriedem idk what you are doing there
17:24:43 sean-k-mooney im using the fake notifiyer to wait for the reboot end notifcaiont
17:25:00 mriedem but why do you need a whole separate _get_server_info
17:25:07 sean-k-mooney the same one that is used in the allocation fucntional tests

Earlier   Later