| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-05-21 | |||
| 14:08:07 | efried | sean-k-mooney: I'll put it in the slot. | |
| 14:11:44 | sean-k-mooney | looks like teh confligt was just in the nova object version file so i should have a rebased version up shortly | |
| 14:12:59 | efried | aspiers: bottom SEV patches are merging, if you want to get those other rebases working. | |
| 14:14:36 | dansmith | efried: are you holding off on the ironic power state one for ironic side review or something? AFAICT, of the nova people who have looked at it, all are either +vote or out at the moment | |
| 14:15:01 | efried | dansmith: Yes, giving SMEs and people with prior reviews a chance to re-ack. | |
| 14:15:07 | efried | Is there some hurry to merge the spec? | |
| 14:15:43 | efried | dansmith: whoops, I actually asked jroll and melwitt to re-look, but I did it in the -placement channel by accident. | |
| 14:16:00 | dansmith | efried: we've been kinda dragging our feet with it and tssurya has been her usual overly polite self.. we discussed in a couple small groups at ptg about it so I just don't want her to continue to have to wait | |
| 14:16:07 | dansmith | efried: okay | |
| 14:16:42 | jroll | I'm +1 on it | |
| 14:16:49 | efried | dansmith: afaict the code patch isn't being held up by the bp approval https://review.opendev.org/#/c/645611/ | |
| 14:16:52 | sean-k-mooney | i have to admit i kind of like running the nova unit tests on my desktop/server | |
| 14:17:37 | dansmith | efried: ... | |
| 14:17:41 | openstackgerrit | sean mooney proposed openstack/nova master: extend libvirt video model support https://review.opendev.org/647733 | |
| 14:23:29 | sean-k-mooney | i will redeploy with ^ later today but the unit and functional test pass locally | |
| 14:23:58 | sean-k-mooney | has anyone used rtox https://pypi.org/project/rtox/ by the way | |
| 14:31:37 | mriedem | low-hanging-fruit bug if someone wants it https://bugs.launchpad.net/nova/+bug/1829877 | |
| 14:31:38 | openstack | Launchpad bug 1829877 in OpenStack Compute (nova) "avoid lazy-lodaing instance.flavor during cold migration" [Low,Confirmed] | |
| 14:31:43 | mriedem | ooo typo | |
| 14:33:33 | dansmith | efried: tomorrow, the top patch set will have been up for 14 days.. presumably that's plenty of time time for anyone to have voiced their opinion, right? | |
| 14:34:03 | efried | dansmith: "top patch set" of what? | |
| 14:34:48 | dansmith | efried: the latest revision of the ironic power state change. Just trying to figure out what "a few days" means in terms of when we should set reminders to make sure that it gets merged and doesn't get ignored again | |
| 14:35:52 | efried | dansmith: Give it a couple hours for melwitt to come on the scene and pick up the IRC nag I left for her. I expect we can merge it by lunchtime. | |
| 14:36:06 | dansmith | alright | |
| 14:38:49 | efried | tssurya: Meanwhile, you should feel pretty confident this is going to go substantially as it stands, if you have the time and desire to crank on the code some :) | |
| 14:57:57 | mriedem | hmm, scatter_gather_all_cells seems to make us lose the request_id on the context we're using | |
| 14:59:54 | mriedem | which makes debugging not so nice | |
| 15:04:54 | mriedem | the RequestContext.from_dict(context.to_dict()) stuff must overwrite the thread local context | |
| 15:05:08 | mriedem | which messes up the logging, even though those copies should be using the same request id | |
| 15:06:36 | dansmith | mriedem: is it just that the threads we spawn to do the scatter don't get the thread-local set to their new context? | |
| 15:10:14 | mriedem | RequestContext __init__ defaults overwrite=True so i'd think each one is updated | |
| 15:10:48 | mriedem | i'm tracing a cold migration in these logs http://logs.openstack.org/56/656656/12/check/nova-multi-cell/6a403ba/controller/logs/screen-n-super-cond.txt.gz#_May_16_19_46_23_815726 with req-bde576dc-5de2-4113-982d-025c65b97e8d which is the request starting in the API | |
| 15:11:01 | mriedem | but after that we hit the scatter/gather and lose the request id | |
| 15:11:28 | mriedem | after that it's using req-66d8138a-530a-4dec-b1cd-9c22e4402dd1 | |
| 15:19:48 | aspiers | efried: yup thanks | |
| 15:31:38 | kashyap` | efried: This was re-proposed (despite me abandoning it): https://review.opendev.org/#/c/660271/ ("Re-propose the spec to allow specifying a list of CPU models") | |
| 15:31:52 | efried | I noticed that | |
| 15:31:55 | efried | haven't looked yet | |
| 15:32:09 | kashyap` | efried: Don't. After technical discussions w/ the libvirt / QEMU CPU modelling maintainers, we've concluded it's not a good idea | |
| 15:32:14 | kashyap` | At least not in its present form. | |
| 15:32:33 | efried | kashyap`: lmk if you want me to -2 bomb the sucker. | |
| 15:32:41 | kashyap` | efried: Hehe, please do. | |
| 15:32:48 | mriedem | cfriesen: ^ | |
| 15:33:00 | efried | kashyap`: save me finding the one you originally abandoned, got it at your fingertips? | |
| 15:33:08 | mriedem | unrelated, it seems odd that we'd fail to stop a libvirt cirros guest after a minute http://logs.openstack.org/56/656656/12/check/nova-multi-cell/6a403ba/compute1/logs/screen-n-cpu.txt.gz#_May_16_19_47_26_017509 | |
| 15:33:10 | kashyap` | (It was approved previously during Stein-time; but I didn't pay sufficient attention to it.) | |
| 15:33:12 | efried | nm, see the link in the comment. | |
| 15:33:48 | kashyap` | mriedem: Noticed it only today? | |
| 15:34:35 | mriedem | noticed yes, but http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Instance%20failed%20to%20shutdown%20in%2060%20seconds.%5C%22%20AND%20tags%3A%5C%22screen-n-cpu.txt%5C%22&from=7d | |
| 15:34:40 | efried | kashyap`: I did one better, I abandoned it. | |
| 15:35:24 | mriedem | if we're burning up to a minute waiting to gracefully kill cirros guests in the gate that could explain in part why some jobs/tests are really slow | |
| 15:35:43 | kashyap` | efried: Thanks! | |
| 15:38:51 | kashyap` | mriedem: "Needs more investigation." (I'm on mobile data in the library; will check tomm early.) | |
| 15:40:44 | mdbooth | efried: https://review.opendev.org/#/admin/groups/540,members Everybody on that list works for RH except Claudiu, John, and Matt. I don't know Claudiu, and I understand mriedem has some administrative difficulties right now :/ | |
| 15:41:21 | mdbooth | efried: So the project is entirely dependent on johnthetubaguy to land stable patches. | |
| 15:41:28 | mriedem | no it's not | |
| 15:41:36 | mriedem | https://review.opendev.org/#/admin/groups/530,members | |
| 15:41:42 | mriedem | there is an #openstack-stable channel | |
| 15:41:49 | mriedem | and people like mtreinish | |
| 15:42:09 | mriedem | i'd also say smcginnis but he's in the same boat as me | |
| 15:42:10 | kashyap` | mriedem: Aren't you referring to release managers group? | |
| 15:42:17 | mdbooth | Ok. John and Matt Treinish | |
| 15:42:18 | kashyap` | I think mdbooth is referring to "stable-core" | |
| 15:42:34 | mriedem | nova-stable-maint's parent group is stable-maint-core | |
| 15:42:44 | mdbooth | kashyap`: Yeah, mriedem was referring to stable-maint-core, who I understand can ack everything | |
| 15:43:03 | mriedem | Alan Pevecalan.pevec@redhat.comChuck Shortchucks@redhat.comFlavio Percocofpercoco@redhat.com so much for that | |
| 15:43:16 | mdbooth | mriedem: Yeah, it's a bit RH loaded :/ | |
| 15:43:20 | mriedem | at some point can RH just extend me a job offer and i can switch sides here and start approving all RH stuff? | |
| 15:43:47 | mdbooth | mriedem: You won't be allowed under trifecta rules ;) | |
| 15:44:06 | cfriesen | kashyap: what was the objection by the libvirt/qemu folks? As you recall, the weakness in the current model is that you can only specify a model on the node, and it affects all guests on that node. | |
| 15:44:12 | aspiers | mriedem: not if another Linux distro nabs you first ;-) | |
| 15:44:32 | mriedem | so pressure the foundation to figure out wtf is going on with huawei developers to be able to work on open source | |
| 15:45:04 | kashyap` | cfriesen: I don't know if that's a "weakness" per-se. But the objection was the we'd need more data; and it is finicky, error-prone. | |
| 15:45:43 | kashyap` | cfriesen: They also asked: Do you have data on why _not_ use the default 'host-model' -- that has the important double-advantage in that, it auto-adds the relevant flags | |
| 15:46:22 | cfriesen | kashyap`: host-model can be an issue if you happen to land on a brand new compute node that has more advanced hardware than most of the other nodes | |
| 15:46:31 | cfriesen | kashyap`: makes it hard to live migrate | |
| 15:47:01 | kashyap` | cfriesen: Not quite -- 'host-model' explicitly makes sure that you _can_ live-migrate across disparate hosts. That's one of the reasons why Nova defaults to it | |
| 15:47:03 | sean-k-mooney | cfriesen: yes that is true but its kind of a know issue | |
| 15:47:27 | cfriesen | sean-k-mooney: yeah, but the whole point of the CPU model feature was to fix that known issue. :) | |
| 15:48:03 | sean-k-mooney | kashyap`: what he is raising is if you have 10 skylake servers and 90 ivybridge servers then if you land on the skylake ones you will only be able to migrate to the skylake ones | |
| 15:48:21 | cfriesen | as it stands, I think you'd need to set up host aggregates for the different hardware generations, or something like that. | |
| 15:48:30 | mriedem | fwiw i've had a non-RH core nom in mind for stable | |
| 15:48:35 | sean-k-mooney | so you would set cpu_mode=custom cpu_mode=ivybridge to mitigate against that | |
| 15:48:46 | mdbooth | mriedem: Do say! | |
| 15:48:51 | mriedem | ill never tell! | |
| 15:49:06 | mriedem | i can start that conversation again with the nova stable maint team | |
| 16:01:05 | sean-k-mooney | cfriesen: that is one apprch yes all though if i ever get arround to it i really want to play with the idea of a boaring host weigher that use traits as a proxy for how intersting a host is and selects the one with the least traits | |
| 16:01:15 | sean-k-mooney | e.g. the most boaring hosts | |
| 16:01:44 | cfriesen | boaring? like wild pigs? :) | |
| 16:01:46 | sean-k-mooney | which woudl mean if you did not ask for a trait specificaly the weigher would prefer selecting the oldest host | |
| 16:01:56 | cfriesen | sean-k-mooney: yeah, that's not a bad idea | |
| 16:02:09 | cfriesen | me too | |
| 16:02:31 | sean-k-mooney | yes its a weigher that digs holes :) | |
| 16:03:19 | cfriesen | sean-k-mooney: you'd still have the problem though that if you happened to land on a "newer" node due to resource pressure you'd be stuck on that hardware for live migration | |
| 16:04:05 | sean-k-mooney | ya that is true but i acutlly want it for other reasons like avoiding my host that have gpus when you did not ask for them that sort of thing | |
| 16:04:05 | cfriesen | it just seems unnecessarily restrictive to say that all guests on a node must have the same virtual CPU | |
| 16:04:48 | sean-k-mooney | well that is what nova has always done to this point but ya im aware of the new spec too | |
| 16:05:27 | sean-k-mooney | personally i just use host-passthough and say screw it ill use host aggrates to seperate things if i need too | |
| 16:06:11 | sean-k-mooney | actully i used ot use AZs at intel for that and ould have one AZ per generation but that was just so i could select the generation simplely when lanuching a vm | |
| 16:06:13 | mtreinish | mriedem: do I need to review something? | |