Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-05-16
16:20:05 cfriesen dansmith: so if you've got 16 ports on an instance...
16:20:45 dansmith melwitt: we've built multiple lists of cells that will have to be flushed, and this is another, is my only point
16:21:06 melwitt ack
16:21:13 dansmith cfriesen: I understand, but that still doesn't mean that neutron should send a single request with 16 identical events for the same instance
16:23:16 cfriesen dansmith: oh, I agree. was just trying to show how we could end up in this scenario since earlier you said you were pretty sure neutron didn't do that.
16:24:18 dansmith cfriesen: I meant for multiple instances
16:25:42 cfriesen dansmith: with the neutron BatchNotifier code I think neutron just sends all queued notifications every couple seconds
16:26:00 cfriesen dansmith: so I think it could include events for multiple instances
16:27:01 dansmith cfriesen: that would be a change from how it originally behaved, and would delay instance booting unnecessarily
16:27:16 dansmith I don't see any timing in that code
16:27:35 cfriesen look at neutron/notifiers/batch_notifier.py
16:28:53 dansmith yeah, that's unfortunate
16:30:09 cfriesen we'll try pushing a neutron change to remove duplicate network-changed events for the same instance from the batch
16:31:00 dansmith in your nova dupe check, you should log a warning or something so we can see if this ever regresses once you fix the sending side
16:31:16 cfriesen was just going to ask if we wanted a nova dupe check as well
16:41:50 sean-k-mooney sfinucan: https://bugs.launchpad.net/nova/+bug/1691178 this is the bug for the qemu cpu affinity ill upload the patch WIP patch in a few minutes but if you have in put on the scheduler question feel free to advise.
16:41:50 openstack Launchpad bug 1691178 in OpenStack Compute (nova) "virt_type=qemu does not support cpu affinity." [Undecided,New] - Assigned to sean mooney (sean-k-mooney)
16:43:00 sean-k-mooney sfinucan: there isnt much in the bug but i will add you to the review of the code change when i split it out of my other changes.
16:47:01 cfriesen sean-k-mooney: would we maybe want to make this configurable? otherwise you're going to need nested kvm to test any of the "dedicated" cpu policy stuff
16:47:16 cfriesen (or real hardware)
16:48:48 sean-k-mooney cfriesen: i was going to make it condtional based on the config value of virt_type
16:49:08 sean-k-mooney cfriesen: cpu pinning is currently only tested in intels nfv ci
16:50:16 sean-k-mooney cfriesen: i basically looks like this https://pastebin.com/afp0M4Rm
16:50:26 sean-k-mooney im currently rebasing to master
16:51:14 sean-k-mooney cfriesen: sfinucan basically just adding if(CONF.libvirt.virt_type != "qemu"): in 2 places in the libvirt dirver and then i need to check unit tests and update if needed
16:53:00 dansmith melwitt: your comment about stale objects is about conf change? or are you worried that seldom-used ones will actually become stale in some un-usable way?
16:54:53 sean-k-mooney cfriesen: sfinucan that change should be in /nova/virt/libvirt/driver.py not /nova/virt/libvirt/config.py ... this is why i never edit diffs/patches
16:55:00 efried I just discovered this (apologies if everyone else knew it already): Add `-a` to journalctl and you get COLORS!
16:56:00 melwitt dansmith: I was thinking more of the latter, though I can't imagine it being much of a problem in practice. it would be things like, removing cells from the fleet or if a cell went temporarily offline
16:56:26 dansmith hmm
16:56:28 melwitt conf change is important but I wasn't thinking of that yet
16:56:40 cfriesen sean-k-mooney: I know at least some of our devs are testing with virtualbox which doesn't do nested KVM. You'd basically require nested KVM or real hardware to exercise the cpu pinning logic. I think a "permissive_qemu_affinity" config option might make sense, though I know we're trying to get rid of config options in general.
16:56:43 dansmith we don't have to do something if we restart the db today, right?
16:56:55 dansmith it reconnects for us I think, so I'm not sure we need to actually worry about things going stale
16:57:03 dansmith and by conf change, I mean changing a cell record or something
16:57:10 melwitt I don't think so, no. good point
16:57:27 melwitt yeah, understood. like changing a connection string
16:57:33 sean-k-mooney cfriesen: you dont need nested virt you just need the kvm module loaded in the vm
16:57:35 dansmith right
16:57:48 cfriesen sean-k-mooney: the kvm module won't load without nested KVM
16:58:07 sean-k-mooney if they are using virt_type=qemu they cannont use any form of cpu pinning becasue TSG does not support it
16:58:23 cfriesen sean-k-mooney: it accepts the commands, it just doesn't work properly. :)
16:58:50 sean-k-mooney libvirt throws an error though whne you try to spawn the vm from nova
16:59:19 cfriesen sean-k-mooney: not ours...wonder what version that was added in
16:59:37 sean-k-mooney cfriesen: i am testing with qemu 2.8
17:00:04 sean-k-mooney i also have libvirt 3.3 form master currently but this happens on older version too
17:00:22 cfriesen sean-k-mooney: I'm on qemu 2.6 and libvirt 2.0
17:01:25 sean-k-mooney cfriesen: im a litlle unfortable adding a new conf option just for this as well if we do they are almost impossible to get rid of
17:01:33 cfriesen sean-k-mooney: we can work around it if necessary
17:02:08 sean-k-mooney cfriesen: do you know is there any way to query this from qemu/libvirt so we could decet at runtime?
17:03:55 cfriesen sean-k-mooney: no clue. :) I'm a bit surprised that your libvirt complains since for the most part (especially for single-vcpu guests) it seems to work okay.
17:06:43 sean-k-mooney cfriesen: i think it is actully qemu that is complaining rather then libvirt it just passes along the message
17:07:39 sean-k-mooney cfriesen: libvirt happilly accpts the xml as it is vailid but TCG then complaines when qemu is starting the instance
17:35:38 openstackgerrit Gage Hugo proposed openstack/nova master: Admin API Policy contingent on is_admin_project https://review.openstack.org/384148
18:04:17 shewless Hi. I have an openstack installation that has a working spice console (I can connect to the console via horizon). I'm trying to connect to the console using "virt-viewer" or some other local client instead of from the web. Any hints on how to do this?
18:04:34 openstackgerrit Gábor Antal proposed openstack/nova master: Transform keypair.delete notification https://review.openstack.org/463946
18:05:08 shewless Basically I think that nova is serving spice via the html viewer over a proxy .. but I want to access the spice server directly using a spice client (to get audio, etc).
18:15:29 sean-k-mooney shewless: i belive 'nova help get-spice-console' will provide the info you are looking for
18:18:17 sean-k-mooney shewless: for my setup at least i dont see a way to connect directly to the spice console though
18:18:58 sean-k-mooney shewless: the different console types available are ['novnc', 'xvpvnc', 'rdp-html5', 'spice-html5', 'serial', 'webmks'] but not just spice
18:27:31 shewless sean-k-mooney: yeah that command gives me the spice url.. but I haven't been able to use my spice client to connect to that url
18:27:40 shewless I think because it's "http" and not "spice" protocol
18:28:51 sean-k-mooney shewless: ya im not sure that nova support direct connection as the client would have to connect directly to spice port on the vm via the compute node
18:29:40 shewless sean-k-mooney: fair enough. Is there a way to get sound to work through the spice html proxy?
18:30:24 jose-phillips hey
18:30:48 sean-k-mooney not that i have found though usally i just use x11 forwarding over ssh or install vnc in the guest and connect to it over its public ip so i have never tried
19:12:33 mriedem smcginnis: https://github.com/openstack/nova/commit/564f14f4bc67463d1ef185909e12ab090f10495b ?
19:13:25 mriedem smcginnis: policy is defined in code here https://github.com/openstack/nova/blob/master/nova/policies/assisted_volume_snapshots.py
19:13:56 smcginnis mriedem: Seeing this: http://logs.openstack.org/51/418451/12/check/gate-tempest-dsvm-full-devstack-plugin-nfs-nv/0f06284/logs/screen-c-vol.txt.gz?level=ERROR#_May_16_08_29_46_647815
19:15:16 smcginnis mriedem: And http://logs.openstack.org/51/418451/12/check/gate-tempest-dsvm-full-devstack-plugin-nfs-nv/0f06284/logs/screen-n-api.txt.gz#_May_16_08_29_46_642168
19:16:12 mriedem smcginnis: https://github.com/openstack/tempest/commit/529eefabc937b6dc6f7b3fae11b5942cc6ee452e
19:16:14 mriedem new tempest change
19:16:21 mriedem tempest doesn't gate on FS-style volume backends
19:16:23 mriedem so that's probably it
19:16:47 mriedem http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Forbidden%3A%20Policy%20doesn't%20allow%20os_compute_api%3Aos-assisted-volume-snapshots%3Acreate%20to%20be%20performed.%20(HTTP%20403)%5C%22&from=7d
19:16:50 mriedem so yeah that's it
19:18:12 armax mriedem: we have a winner
19:18:13 armax http://logs.openstack.org/25/465125/1/experimental/gate-tempest-dsvm-neutron-pg-full-ubuntu-xenial/2a7a080/
19:18:32 mriedem armax: sweet
19:19:07 armax mriedem: nice one
19:19:14 armax mriedem: you need to add a test now ;)
19:19:24 mriedem armax: yup, working on it
19:19:28 armax mriedem: you rock
19:19:40 mriedem smcginnis distracted me with all of his NFS love
19:22:14 armax mriedem: do you want to rerun the job to confirm if there’s another race lurking?
19:22:40 armax mriedem: I thought you learned the craft of multitasking when you became PTL
19:23:41 mriedem armax: i can run it again yeah while i work on tests
19:24:17 armax mriedem: does Zuul listens to one whilst there’s still another ongoing run?
19:24:33 armax oh
19:24:35 armax never mind
19:24:44 mriedem armax: i didn't see that you checked experimental before i did
19:24:50 mriedem so i'll have to do it again once that run is done i think
19:24:56 armax didn’t notice that jenkins came back, we’re good
19:24:59 mriedem artom_: let me know if i'm off base here https://review.openstack.org/#/c/415520/
19:34:23 mriedem armax: do we need to worry about the subnet being gone too?
19:34:32 mriedem L450
19:35:34 openstackgerrit sean mooney proposed openstack/nova master: do not set vcpu_pin elements when virt_type=qemu https://review.openstack.org/465160
19:38:23 artom_ mriedem, will do, thanks for the comments!
19:39:45 mriedem artom: i think we should just get the device metadata first, then attach to the guest, and then if that's fine, do instance.save() to update the db
19:39:57 mriedem then we don't have to worry about rolling back the detach in the guest

Earlier   Later