| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-27 | |||
| 14:20:00 | sean-k-mooney | ya from a down stream perspecitve we are going ot have to test this carfully for FFU also | |
| 14:20:34 | aspiers | I think it would be good enough for now if we call it once per arch, using the default machine type for that arch specified by hw_machine_type | |
| 14:21:27 | kashyap | aspiers: Yeah. How many arches is SEV supported, BTW? | |
| 14:21:28 | aspiers | later we could consider calling it for other machine types | |
| 14:21:28 | sean-k-mooney | mriedem: what im hoping to do is get the patches inplace next week so we can do some addtional testing before wew fully decided if we will apply the patch. thanks for clarifing :) | |
| 14:21:38 | aspiers | kashyap: only x86_64 I think | |
| 14:22:06 | kashyap | aspiers: Thought as much. Yeah, for now, let's go the x86_64 route, document it in the conf file / wherver appropriate | |
| 14:22:22 | aspiers | kashyap: so the memoized _domaincaps would be a nested dict of dicts like I mentioned the other day | |
| 14:22:28 | sean-k-mooney | aspiers: you can set the machive type in the image_metadata too | |
| 14:22:37 | kashyap | sean-k-mooney: Yes, that's what I'm getting at | |
| 14:23:01 | aspiers | sean-k-mooney: sure, but right now there is nothing in domain capabilities which would need to influence that | |
| 14:23:05 | kashyap | $ openstack image set \ | |
| 14:23:06 | kashyap | --property hw_machine_type=x86_64=q35 Fedora-29-Template | |
| 14:23:20 | kashyap | aspiers: Aside, ^ have you tested the above with Git master? | |
| 14:23:50 | aspiers | kashyap: have I tested setting image props? | |
| 14:23:52 | kashyap | Yes | |
| 14:23:55 | aspiers | no | |
| 14:23:58 | kashyap | (Asking because, I got a bug report, albiet for an older release, that the above isn't working. I'm still suspicious of the bug report.) | |
| 14:24:03 | aspiers | ah OK | |
| 14:24:20 | kashyap | I asked the reporter to get me the logs, and libvirt debug log (with filters), etc. So I can see what's going on. | |
| 14:24:53 | aspiers | kashyap: so I'm suggesting something like (effectively) self._domaincaps['x86_64']['q35'] = <XML tree object> | |
| 14:24:54 | kashyap | "Seeing is believing." Because too many times I got burnt by taking the word of the bug reporter | |
| 14:25:01 | aspiers | haha yeah :) | |
| 14:25:08 | kashyap | ... only to see in the logs that they misconfigured or did a blatant no-op | |
| 14:26:09 | openstackgerrit | Lajos Katona proposed openstack/python-novaclient master: Add support for microversion v2.70 https://review.openstack.org/637234 | |
| 14:26:27 | kashyap | aspiers: BTW, just saw the comment in the review (and the bug: https://bugzilla.redhat.com/show_bug.cgi?id=1683471) | |
| 14:26:28 | openstack | bugzilla.redhat.com bug 1683471 in libvirt "getDomainCapabilities claims SEV is supported for pc-i440fx-1.4 machine type" [Unspecified,New] - Assigned to libvirt-maint | |
| 14:26:47 | sean-k-mooney | kashyap: i got libvirtError: XML error: No PCI buses available | |
| 14:26:53 | kashyap | (I see you filed it yourself, though) | |
| 14:26:57 | openstackgerrit | Lajos Katona proposed openstack/python-novaclient master: Add support for microversion v2.71 https://review.openstack.org/637234 | |
| 14:26:57 | sean-k-mooney | when i tried that a few seconds ago | |
| 14:27:01 | kashyap | sean-k-mooney: Ah, on Git master? | |
| 14:27:07 | kashyap | sean-k-mooney: So something is broken :-( | |
| 14:27:15 | aspiers | kashyap: yes :) | |
| 14:27:24 | kashyap | sean-k-mooney: Can you add the log filters I noted in the second bullet here: | |
| 14:27:27 | kashyap | https://kashyapc.fedorapeople.org/virt/openstack/request-nova-libvirt-qemu-debug-logs.txt | |
| 14:27:37 | kashyap | sean-k-mooney: And then re-post the complete libvirt debug log somewhere, if you can... | |
| 14:27:41 | sean-k-mooney | on f77791954d8a39652ea30ccf51968886471de612 form master | |
| 14:27:46 | aspiers | kashyap: does that dict of dicts make sense to you? it allows for calling getDomCaps on more machine types in the future | |
| 14:28:02 | aspiers | and at least one per arch for now | |
| 14:29:04 | kashyap | aspiers: Yeah, it does make sense to me. (As we future-proofed it.) | |
| 14:29:39 | aspiers | kashyap: OK great, I'll update the patch to do that and test it on a real SEV machine | |
| 14:29:51 | sean-k-mooney | kashyap: the only difference form the default filter on ubunut is the also have 1:cpu | |
| 14:30:30 | kashyap | sean-k-mooney: Did you get your default filter from DevStack? (I made sure to add '1:cpu' there too) | |
| 14:30:42 | sean-k-mooney | yes | |
| 14:31:04 | kashyap | aspiers: Thanks! I've got the SEV series in my browser open. | |
| 14:31:18 | kashyap | ("Open in the browser" doesn't mean much, though :D) | |
| 14:31:25 | sean-k-mooney | at least i assume so since this was just a ubunut cloud image so anything not stadard was devstack | |
| 14:32:00 | kashyap | sean-k-mooney: If you got it from DevStack, you should have '1:cpu'. | |
| 14:32:07 | kashyap | Because I added: | |
| 14:32:08 | kashyap | lib/nova_plugins/functions-libvirt: local log_filters="1:libvirt.c 1:qemu 1:conf 1:security 3:object 3:event 3:json 3:file 1:util 1:cpu" | |
| 14:32:12 | kashyap | (In DevStack) | |
| 14:32:45 | aspiers | kashyap: LOL :) maybe you need this https://chrome.google.com/webstore/detail/xtab/amddgdnlkmohapieeekfknakgdnpbleb | |
| 14:32:48 | sean-k-mooney | kashyap: yep i have but the string in https://kashyapc.fedorapeople.org/virt/openstack/request-nova-libvirt-qemu-debug-logs.txt does not have 1:cpu | |
| 14:32:59 | gibi | jaypipes, efried: the fup for the bandwidth series is up-to-date and mriedem already +2 it | |
| 14:33:03 | gibi | https://review.openstack.org/#/c/639159/ | |
| 14:33:09 | jaypipes | ack, on it. | |
| 14:33:14 | gibi | jaypipes: thanks a lot | |
| 14:33:23 | kashyap | sean-k-mooney: Heh, then I should update my doc :-) | |
| 14:33:34 | kashyap | sean-k-mooney: Okay, we're good. | |
| 14:34:05 | kashyap | aspiers: Haha, seems dangerous. As it destroys the illusion of productivity! | |
| 14:34:41 | aspiers | X-D | |
| 14:35:56 | aspiers | on a tangent, I wonder if I broke a world record by having 89 Chrome extensions installed | |
| 14:37:57 | sean-k-mooney | kashyap: looking at the libvirtd log there isnt really any useful infor there. just the same message. this is more then likely a nova issue not a libvirt one | |
| 14:39:06 | jaypipes | gibi: all done. | |
| 14:39:07 | kashyap | sean-k-mooney: Yeah, could be the way Nova is generating guest XML too | |
| 14:39:09 | sean-k-mooney | kashyap: this is the section fo the log around the issu http://paste.openstack.org/show/746435/ | |
| 14:39:18 | kashyap | sean-k-mooney: Anything fun in the n-cpu.log? | |
| 14:39:55 | gibi | jaypipes: thanks. The rest of the series also up-to-date if you have time https://review.openstack.org/#/c/639653 | |
| 14:39:58 | sean-k-mooney | well there was a traceback but ill check the xml that was generated on second while i grab it | |
| 14:40:55 | kashyap | sean-k-mooney: Thanks! | |
| 14:41:30 | kashyap | sean-k-mooney: Care to post the full libvirtd.log just somewhere? I'd like to see some messages above that too | |
| 14:42:04 | sean-k-mooney | http://paste.openstack.org/show/746436/ | |
| 14:42:38 | sean-k-mooney | am sure i can but its got a lot of crap in it since just checked it in one of my long running dev vms | |
| 14:43:47 | sean-k-mooney | kashyap: looking at the xml we are not generating a pcie bus or adding a pci bridge device | |
| 14:43:55 | kashyap | sean-k-mooney: Ah, right; I normally just clean it up `> foo.log` and start over | |
| 14:44:36 | sean-k-mooney | i can truncate it can spin up another vm it will only take a second but this looks like a nova issue | |
| 14:44:58 | mriedem | gibi: i know you told me to leave you alone, but i'm a soft -1 on the bottom 2 changes in the series now, should be easy things to address and then i can +2 those today | |
| 14:45:08 | sean-k-mooney | for q35 we expect nova to generate a pcie toplogy in the xml | |
| 14:45:43 | kashyap | sean-k-mooney: Hang on ... | |
| 14:46:17 | kashyap | sean-k-mooney: This is not even valid guest XML: | |
| 14:46:19 | kashyap | <type machine="x86_64=q35">hvm</type> | |
| 14:46:30 | sean-k-mooney | hehe ya i just noticed that too | |
| 14:47:01 | kashyap | sean-k-mooney: Also, it's a bug in libvirt itself! | |
| 14:47:07 | kashyap | sean-k-mooney: It should reject that as an invalid XML | |
| 14:47:17 | sean-k-mooney | it did | |
| 14:47:20 | kashyap | I'll file an upstream bug report, and take it w/ the libvirt folks | |
| 14:47:34 | openstackgerrit | Yongli He proposed openstack/nova master: Adds the server group info into show server detail API. https://review.openstack.org/621474 | |
| 14:48:03 | sean-k-mooney | the error startswith "Error defining a guest with XML: ..." | |
| 14:48:14 | sean-k-mooney | maybe that came form nova | |
| 14:48:26 | kashyap | I guess so; libvirt uses the terms "domains" | |
| 14:48:34 | kashyap | sean-k-mooney: Regardless, libvirt's parser should reject invalid XML | |
| 14:48:36 | sean-k-mooney | ah right | |
| 14:48:49 | kashyap | It is a valid bug in libvirt itself; I just double-confirmed w/ a libvirt upstream dev. | |
| 14:48:58 | sean-k-mooney | yep let me quickly set the image prop to q35 | |
| 14:50:01 | kashyap | sean-k-mooney: So, this is valid guest XML: <type arch='x86_64' machine='pc-i440fx-rhel7.2.0'>hvm</type> | |
| 14:50:42 | sean-k-mooney | kashyap: openstack image set --property hw_machine_type=q35 cirros-0.3.6-x86_64-disk works | |
| 14:51:04 | kashyap | sean-k-mooney: Excellent. So the problem is the bug reporter directly copied my bad syntax! | |
| 14:51:12 | kashyap | Talk about copy/paste on the internet. | |
| 14:51:24 | sean-k-mooney | http://paste.openstack.org/show/746437/ | |