| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-27 | |||
| 14:26:53 | kashyap | (I see you filed it yourself, though) | |
| 14:26:57 | sean-k-mooney | when i tried that a few seconds ago | |
| 14:26:57 | openstackgerrit | Lajos Katona proposed openstack/python-novaclient master: Add support for microversion v2.71 https://review.openstack.org/637234 | |
| 14:27:01 | kashyap | sean-k-mooney: Ah, on Git master? | |
| 14:27:07 | kashyap | sean-k-mooney: So something is broken :-( | |
| 14:27:15 | aspiers | kashyap: yes :) | |
| 14:27:24 | kashyap | sean-k-mooney: Can you add the log filters I noted in the second bullet here: | |
| 14:27:27 | kashyap | https://kashyapc.fedorapeople.org/virt/openstack/request-nova-libvirt-qemu-debug-logs.txt | |
| 14:27:37 | kashyap | sean-k-mooney: And then re-post the complete libvirt debug log somewhere, if you can... | |
| 14:27:41 | sean-k-mooney | on f77791954d8a39652ea30ccf51968886471de612 form master | |
| 14:27:46 | aspiers | kashyap: does that dict of dicts make sense to you? it allows for calling getDomCaps on more machine types in the future | |
| 14:28:02 | aspiers | and at least one per arch for now | |
| 14:29:04 | kashyap | aspiers: Yeah, it does make sense to me. (As we future-proofed it.) | |
| 14:29:39 | aspiers | kashyap: OK great, I'll update the patch to do that and test it on a real SEV machine | |
| 14:29:51 | sean-k-mooney | kashyap: the only difference form the default filter on ubunut is the also have 1:cpu | |
| 14:30:30 | kashyap | sean-k-mooney: Did you get your default filter from DevStack? (I made sure to add '1:cpu' there too) | |
| 14:30:42 | sean-k-mooney | yes | |
| 14:31:04 | kashyap | aspiers: Thanks! I've got the SEV series in my browser open. | |
| 14:31:18 | kashyap | ("Open in the browser" doesn't mean much, though :D) | |
| 14:31:25 | sean-k-mooney | at least i assume so since this was just a ubunut cloud image so anything not stadard was devstack | |
| 14:32:00 | kashyap | sean-k-mooney: If you got it from DevStack, you should have '1:cpu'. | |
| 14:32:07 | kashyap | Because I added: | |
| 14:32:08 | kashyap | lib/nova_plugins/functions-libvirt: local log_filters="1:libvirt.c 1:qemu 1:conf 1:security 3:object 3:event 3:json 3:file 1:util 1:cpu" | |
| 14:32:12 | kashyap | (In DevStack) | |
| 14:32:45 | aspiers | kashyap: LOL :) maybe you need this https://chrome.google.com/webstore/detail/xtab/amddgdnlkmohapieeekfknakgdnpbleb | |
| 14:32:48 | sean-k-mooney | kashyap: yep i have but the string in https://kashyapc.fedorapeople.org/virt/openstack/request-nova-libvirt-qemu-debug-logs.txt does not have 1:cpu | |
| 14:32:59 | gibi | jaypipes, efried: the fup for the bandwidth series is up-to-date and mriedem already +2 it | |
| 14:33:03 | gibi | https://review.openstack.org/#/c/639159/ | |
| 14:33:09 | jaypipes | ack, on it. | |
| 14:33:14 | gibi | jaypipes: thanks a lot | |
| 14:33:23 | kashyap | sean-k-mooney: Heh, then I should update my doc :-) | |
| 14:33:34 | kashyap | sean-k-mooney: Okay, we're good. | |
| 14:34:05 | kashyap | aspiers: Haha, seems dangerous. As it destroys the illusion of productivity! | |
| 14:34:41 | aspiers | X-D | |
| 14:35:56 | aspiers | on a tangent, I wonder if I broke a world record by having 89 Chrome extensions installed | |
| 14:37:57 | sean-k-mooney | kashyap: looking at the libvirtd log there isnt really any useful infor there. just the same message. this is more then likely a nova issue not a libvirt one | |
| 14:39:06 | jaypipes | gibi: all done. | |
| 14:39:07 | kashyap | sean-k-mooney: Yeah, could be the way Nova is generating guest XML too | |
| 14:39:09 | sean-k-mooney | kashyap: this is the section fo the log around the issu http://paste.openstack.org/show/746435/ | |
| 14:39:18 | kashyap | sean-k-mooney: Anything fun in the n-cpu.log? | |
| 14:39:55 | gibi | jaypipes: thanks. The rest of the series also up-to-date if you have time https://review.openstack.org/#/c/639653 | |
| 14:39:58 | sean-k-mooney | well there was a traceback but ill check the xml that was generated on second while i grab it | |
| 14:40:55 | kashyap | sean-k-mooney: Thanks! | |
| 14:41:30 | kashyap | sean-k-mooney: Care to post the full libvirtd.log just somewhere? I'd like to see some messages above that too | |
| 14:42:04 | sean-k-mooney | http://paste.openstack.org/show/746436/ | |
| 14:42:38 | sean-k-mooney | am sure i can but its got a lot of crap in it since just checked it in one of my long running dev vms | |
| 14:43:47 | sean-k-mooney | kashyap: looking at the xml we are not generating a pcie bus or adding a pci bridge device | |
| 14:43:55 | kashyap | sean-k-mooney: Ah, right; I normally just clean it up `> foo.log` and start over | |
| 14:44:36 | sean-k-mooney | i can truncate it can spin up another vm it will only take a second but this looks like a nova issue | |
| 14:44:58 | mriedem | gibi: i know you told me to leave you alone, but i'm a soft -1 on the bottom 2 changes in the series now, should be easy things to address and then i can +2 those today | |
| 14:45:08 | sean-k-mooney | for q35 we expect nova to generate a pcie toplogy in the xml | |
| 14:45:43 | kashyap | sean-k-mooney: Hang on ... | |
| 14:46:17 | kashyap | sean-k-mooney: This is not even valid guest XML: | |
| 14:46:19 | kashyap | <type machine="x86_64=q35">hvm</type> | |
| 14:46:30 | sean-k-mooney | hehe ya i just noticed that too | |
| 14:47:01 | kashyap | sean-k-mooney: Also, it's a bug in libvirt itself! | |
| 14:47:07 | kashyap | sean-k-mooney: It should reject that as an invalid XML | |
| 14:47:17 | sean-k-mooney | it did | |
| 14:47:20 | kashyap | I'll file an upstream bug report, and take it w/ the libvirt folks | |
| 14:47:34 | openstackgerrit | Yongli He proposed openstack/nova master: Adds the server group info into show server detail API. https://review.openstack.org/621474 | |
| 14:48:03 | sean-k-mooney | the error startswith "Error defining a guest with XML: ..." | |
| 14:48:14 | sean-k-mooney | maybe that came form nova | |
| 14:48:26 | kashyap | I guess so; libvirt uses the terms "domains" | |
| 14:48:34 | kashyap | sean-k-mooney: Regardless, libvirt's parser should reject invalid XML | |
| 14:48:36 | sean-k-mooney | ah right | |
| 14:48:49 | kashyap | It is a valid bug in libvirt itself; I just double-confirmed w/ a libvirt upstream dev. | |
| 14:48:58 | sean-k-mooney | yep let me quickly set the image prop to q35 | |
| 14:50:01 | kashyap | sean-k-mooney: So, this is valid guest XML: <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> | |
| 14:50:42 | sean-k-mooney | kashyap: openstack image set --property hw_machine_type=q35 cirros-0.3.6-x86_64-disk works | |
| 14:51:04 | kashyap | sean-k-mooney: Excellent. So the problem is the bug reporter directly copied my bad syntax! | |
| 14:51:12 | kashyap | Talk about copy/paste on the internet. | |
| 14:51:24 | sean-k-mooney | http://paste.openstack.org/show/746437/ | |
| 14:51:49 | sean-k-mooney | kashyap: well the syntax you used was correct for the nova.conf | |
| 14:52:21 | sean-k-mooney | but an image presuably only has on architeure so i guess droping the arch= part makes sense | |
| 14:52:47 | kashyap | Yeah | |
| 14:53:11 | kashyap | sean-k-mooney: Thanks for the debugging; two birds, one stone. ;-) | |
| 14:53:23 | yonglihe | Surya's comments addressed. thanks. https://review.openstack.org/#/c/621474/27 | |
| 14:53:37 | sean-k-mooney | you could argue there is also a nova bug here in that nova did not validate the image metadata property before passing it to libvirt | |
| 14:54:22 | kashyap | sean-k-mooney: That, too. | |
| 14:54:51 | yonglihe | jaypipes: topology patch rebase to microversion 2.72 https://review.openstack.org/#/c/621476/ | |
| 14:59:05 | mriedem | gibi: -1 on https://review.openstack.org/#/c/625310/ as well | |
| 15:00:58 | kashyap | sean-k-mooney: Can you tell the libvirt/QEMU version you tested with, please? | |
| 15:01:04 | mriedem | stephenfin: you should probably take a look at this before me https://review.openstack.org/#/c/625311/ | |
| 15:03:31 | sean-k-mooney | ubuntu@os-vif-1:/opt/stack/nova$ qemu-system-x86_64 --version | |
| 15:03:32 | sean-k-mooney | QEMU emulator version 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.9) | |
| 15:03:34 | sean-k-mooney | Copyright (c) 2003-2017 Fabrice Bellard and the QEMU Project developers | |
| 15:03:35 | cfriesen | mriedem: not sure what you mean by this comment: https://review.openstack.org/#/c/620706/25/nova/compute/api.py@612 | |
| 15:03:36 | sean-k-mooney | ubuntu@os-vif-1:/opt/stack/nova$ libvirtd --version | |
| 15:03:38 | sean-k-mooney | libvirtd (libvirt) 4.0.0 | |
| 15:04:52 | mriedem | cfriesen: reference to https://review.openstack.org/#/c/620706/25/nova/compute/api.py@3569 | |
| 15:06:36 | mriedem | cfriesen: left another comment | |
| 15:06:44 | mriedem | i'd care less about that if it wasn't copy/paste | |
| 15:06:49 | mriedem | DRY that up | |
| 15:06:50 | mriedem | please | |
| 15:14:55 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif master: Add "master" parameter to ip.set() API function https://review.openstack.org/639702 | |
| 15:15:50 | sean-k-mooney | jaypipes: stephenfin by the way i missed ^ in the brctl removal so we need to respin os-vif... | |
| 15:15:52 | jaypipes | yonglihe: yes, I have that in my queue. I have major concerns about the response format in that patch. It is mixing the concept of a guest's virtual CPU topology with that of the NUMA information, and the two things don't really have anything to do with each other -- and unfortunately the way that InstanceNUMATopology and InstanceNUMACell objects were designed, that bad coupling is now going to leak out through the public REST API, which is a | |
| 15:15:52 | jaypipes | problem I raised during the spec review I believe. | |
| 15:24:29 | cfriesen | mriedem: regarding https://review.openstack.org/#/c/620706/25/nova/compute/api.py@3569 if I do that then wouldn't I need to change the existing code in _validate_flavor_image() where it's doing stuff like image.get('properties', {})? | |
| 15:29:27 | stephenfin | mriedem, gibi: Reviewed https://review.openstack.org/#/c/625311/ Two main issues plus a lot of doc nits (which can be ignored for now) | |