| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-09-04 | |||
| 17:47:45 | stephenfin | It's a similar thing to what you need, I suspect | |
| 17:47:52 | stephenfin | sean-k-mooney can confirm | |
| 17:47:54 | sean-k-mooney | aspiers: what i would do is write a check in hardware.py | |
| 17:47:57 | stephenfin | though /me has to go | |
| 17:47:58 | sean-k-mooney | then call that form the api | |
| 17:48:02 | sean-k-mooney | and or conductor | |
| 17:48:09 | aspiers | sean-k-mooney: there is already a check in hardware.py | |
| 17:48:32 | sean-k-mooney | aspiers: well we use check in hardware.py from the api for other things | |
| 17:48:54 | mriedem | fwiw i don't see any other virt driver supporting sev anytime soon | |
| 17:49:03 | aspiers | mriedem: agreed | |
| 17:49:13 | mriedem | the vmware driver is basically maintenance mode | |
| 17:49:22 | mriedem | xen is on the way out | |
| 17:49:23 | aspiers | sean-k-mooney: https://opendev.org/openstack/nova/src/branch/master/nova/virt/hardware.py#L1140 | |
| 17:49:30 | mriedem | oh that reminds me, | |
| 17:49:38 | mriedem | efried: should we formally deprecate the xen driver before releasing train? | |
| 17:49:47 | dansmith | wow | |
| 17:49:51 | mriedem | right now you just get a quality warning in the logs on startup | |
| 17:50:05 | mriedem | dansmith: we talked about that at the ptg | |
| 17:50:11 | mriedem | hasn't had CI in probably a year | |
| 17:50:17 | dansmith | I know | |
| 17:50:23 | sean-k-mooney | mriedem: didnt we say we would at the PTG | |
| 17:50:25 | dansmith | it's just...a big and sad deal | |
| 17:50:40 | sean-k-mooney | mriedem: well if nothing impove in train | |
| 17:50:49 | mriedem | deprecating doesn't mean we remove in U | |
| 17:50:52 | mriedem | just that we have the option | |
| 17:50:57 | dansmith | yup | |
| 17:51:01 | sean-k-mooney | mriedem: i dont think testing has impvoed so we proably sue | |
| 17:51:04 | mriedem | we still need to nix nova-net | |
| 17:51:07 | sean-k-mooney | shoudl | |
| 17:51:16 | dansmith | deprecation should be a shot across the bow.. if nobody steps up at that point, then.. the signal is clear | |
| 17:51:40 | mriedem | i'll put it on the meeting agenda for tomorrow so it's in the meeting logs | |
| 17:51:41 | mriedem | and such | |
| 17:51:47 | mriedem | such and such and so and so | |
| 17:52:02 | sean-k-mooney | aspiers: ya so i would just call get_mem_encryption_constraint form the api and reject the live migration if its true | |
| 17:52:16 | donnyd | hasn't nova-net been on the chopping block for the better half of a decade? | |
| 17:52:25 | aspiers | sean-k-mooney: right, so is nova.compute.api.API.live_migrate() the right method to do the rejection? | |
| 17:52:30 | mriedem | donnyd: deprecated in newton | |
| 17:52:44 | aspiers | sean-k-mooney: and nova.compute.api.API.suspend()? | |
| 17:52:47 | mriedem | donnyd: nova-net gets hard to remove when it's tendrils are in everything | |
| 17:52:53 | mriedem | *its | |
| 17:53:07 | donnyd | makes sense | |
| 17:53:25 | melwitt | donnyd: did you ever figure out your slow image download problem? I had talked to penick about it and he said with the hardware specs you gave, download should be fast (like you were also thinking) | |
| 17:53:39 | donnyd | no | |
| 17:53:43 | donnyd | its still slow | |
| 17:53:59 | melwitt | have you tried a simple scp from machine to machine to see if that's similarly slow? he suggested that | |
| 17:54:18 | melwitt | to narrow down whether it's related to nova or not | |
| 17:55:32 | dansmith | I haven't followed this issue, but an scp can be cpu bound and not max out a fast connection | |
| 17:55:54 | dansmith | if it's really bad, then it going faster will tell you something, | |
| 17:55:56 | dansmith | but if it's 10gE or something, generally iperf or straight netcat will help you go faste | |
| 17:56:00 | dansmith | *faster | |
| 17:56:20 | melwitt | oh, ok didn't know that | |
| 17:56:28 | sean-k-mooney | aspiers: you could add a decorator and ecorate those metods but im not sure what the normal procedure would be | |
| 17:56:50 | aspiers | sean-k-mooney: yes I'm writing a decorator as per mriedem's suggestion but I want to make sure I'm decorating the right methods :) | |
| 17:56:50 | sean-k-mooney | for cyborge i think we chose not to do this | |
| 17:56:56 | aspiers | sean-k-mooney: in the right file | |
| 17:57:02 | mriedem | sean-k-mooney: there isn't really a normal procedure, | |
| 17:57:07 | sean-k-mooney | we chose to jsut document what ws not supprotted | |
| 17:57:11 | donnyd | scp openstack:/var/lib/glance/images/8dd685fd-277d-46fb-8d1c-b05ff7c5ecaa | |
| 17:57:12 | donnyd | 8dd685fd-277d-46fb-8d1c-b05ff7c5ecaa 100% 6212MB 209.1MB/s 00:29 | |
| 17:57:16 | mriedem | most times when we know shit doesn't work we just don't fix it, don't fail fast detect it, or document it | |
| 17:57:24 | donnyd | so 2x faster than glance | |
| 17:57:38 | mriedem | over time people eventually get tired of that so they put fail fast checks in place, like the things that don't work in the api with volume-backed servers | |
| 17:57:40 | donnyd | but dansmith is correct. CPU bound on the controller side | |
| 17:58:15 | sean-k-mooney | mriedem: yes so aspiers is breakign the mold by puting the check in place with the code | |
| 17:58:16 | dansmith | donnyd: scp'ing from and to the same disks on each side presumably? | |
| 17:58:42 | mriedem | sean-k-mooney: not really, like i said, we have "this doesn't work with this type of instance" checks in the API for certain things | |
| 17:58:56 | mriedem | he's just doing it a bit differently with a decorator, it's the same idea though | |
| 17:59:14 | sean-k-mooney | ya | |
| 17:59:28 | sean-k-mooney | i understand that and that we do have check like this | |
| 17:59:41 | sean-k-mooney | aspiers: so what are you going to return to the use | |
| 17:59:47 | sean-k-mooney | a 4** | |
| 17:59:54 | mriedem | the bw provider qos stuff fails fast in the api for move ops today too | |
| 18:00:01 | sean-k-mooney | yes | |
| 18:00:10 | sean-k-mooney | i was looking ot see if i could find that | |
| 18:00:20 | donnyd | https://www.irccloud.com/pastebin/wQsy8Dyc/ | |
| 18:00:35 | donnyd | not likely to be a network issue | |
| 18:00:47 | mriedem | sean-k-mooney: for the bw provider case (supports_port_resource_request_during_move) we return a 400 | |
| 18:00:58 | sean-k-mooney | for consitnece we proablyt want to fail the migration with the same respoce code | |
| 18:01:07 | dansmith | donnyd: there are various chunk sizes for the glance backends, which may be too small by default for a fast network | |
| 18:01:13 | sean-k-mooney | ok 400 bad request makes sense i guess | |
| 18:01:19 | mriedem | i'm not sure that 400 was really correct there, 409 might have been better, but ... | |
| 18:02:17 | aspiers | sean-k-mooney: I can't even find the place where migration chooses which error codes to return. There's no mention of migration in nova.api.openstack.compute.servers | |
| 18:02:20 | sean-k-mooney | i was just looking at 409 | |
| 18:02:21 | sean-k-mooney | https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/409 | |
| 18:02:39 | donnyd | dansmith: using file backend because it performs the fastest | |
| 18:02:41 | sean-k-mooney | i think we had this conversation for bandwidth too | |
| 18:03:07 | aspiers | oh found it | |
| 18:03:07 | dansmith | donnyd: yeah, there are chunk size knobs for several of them, filesystem included | |
| 18:03:17 | aspiers | nova.api.openstack.compute.migrate_servers | |
| 18:03:25 | dansmith | donnyd: 64k by default, apparently, which is pretty dang small | |
| 18:03:49 | donnyd | Yea that is a little on the small side | |
| 18:04:07 | dansmith | for 1gE that's probably okay, but not 10gE I'd think.. | |
| 18:04:23 | sean-k-mooney | aspiers: right here https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/migrate_server.py#L61-L68 | |
| 18:04:29 | donnyd | the controller side is 40G | |
| 18:04:34 | aspiers | OK so I want the check to go in nova.api.openstack.compute.migrate_servers not nova.compute.api | |
| 18:04:36 | donnyd | and each compute is 10 | |
| 18:04:44 | dansmith | donnyd: the rbd default chunk is 8MiB, fwiw | |
| 18:05:15 | sean-k-mooney | aspiers: or in your case you want the live_migrate check below it | |
| 18:05:18 | sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/migrate_server.py#L127-L137 | |
| 18:05:22 | dansmith | although it's not clear if that chunk size affects streaming like the filesystem one does | |