| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-05-28 | |||
| 09:16:23 | sean-k-mooney | os-vif just need to use the constratis file | |
| 09:16:31 | ralonsoh | sean-k-mooney, that's right | |
| 09:16:39 | ralonsoh | sean-k-mooney, I don't need to modify requirements | |
| 09:16:56 | sean-k-mooney | right just backport that patch | |
| 09:16:59 | ralonsoh | sean-k-mooney, I'll submit a patch for os-vif today modifying tox.ini | |
| 09:17:01 | sean-k-mooney | i whould have done that when i wrote it | |
| 09:17:05 | ralonsoh | sean-k-mooney, thanks! | |
| 09:17:31 | sean-k-mooney | can you submit it to all active stable brances | |
| 09:18:01 | ralonsoh | sean-k-mooney, hahahaha | |
| 09:18:03 | ralonsoh | sean-k-mooney, https://review.opendev.org/#/c/658780/ | |
| 09:18:26 | sean-k-mooney | right i had tought you had done this already | |
| 09:18:32 | mrch_ | anyone tapped into ^10:47 | |
| 09:18:36 | ralonsoh | sean-k-mooney, my short memory | |
| 09:20:46 | sean-k-mooney | ralonsoh: it looks like you did not rebase ontop of the constratists fix when you updated https://review.opendev.org/#/c/655694/7 | |
| 09:21:01 | ralonsoh | sean-k-mooney, you are right | |
| 09:21:22 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/os-vif stable/queens: Prevent "qbr" Linux Bridge from replying to ARP messages https://review.opendev.org/655694 | |
| 09:25:21 | openstackgerrit | Balazs Gibizer proposed openstack/os-traits master: Add REPORT_PARENT_INTERFACE_NAME_FOR_SRIOV_NIC trait https://review.opendev.org/658852 | |
| 09:30:05 | gibi | lyarwood: hi! I've replied in https://review.opendev.org/#/c/512553/12//COMMIT_MSG@12 If we could agree with the way forward then I can redo the patch series | |
| 09:31:45 | lyarwood | gibi: ack thanks, I'll check shortly | |
| 09:34:36 | kashyap | lyarwood: Do you also have an opinion on the question I asked in the commit message: https://review.opendev.org/#/c/661574/ ("Document mitigation for Intel MDS security flaws") | |
| 09:34:42 | kashyap | (Thanks for the review, BTW) | |
| 09:39:10 | lyarwood | kashyap: replied, lets open another review for the meltdown doc if you already have something written. | |
| 09:39:42 | kashyap | lyarwood: Yeah, it would be another change indeed. Just wanted to make sure if we _want_ to do it or not. | |
| 09:40:57 | lyarwood | kashyap: I can't think why we wouldn't if you already have some text etc | |
| 09:41:14 | lyarwood | gibi: replied, I'd be happy with that approach. | |
| 09:41:25 | kashyap | lyarwood: Heh, yeah. (The text has been sitting there for a while) | |
| 09:43:09 | gibi | lyarwood: thanks. I will re-shuffle the changes accordingly | |
| 12:33:25 | openstackgerrit | Merged openstack/nova stable/stein: [stable-only] Improve the reno for consoleauth upgrade check https://review.opendev.org/661123 | |
| 12:35:48 | artom | cdent, you know, you'd think with an extensive portfolio of NFV customer who use NUMA but seldom participate upstream, you'd think RH would have a way of asking them en masse about their preferred use cases | |
| 12:36:23 | cdent | one would hope so, but it seems hard for everyone, not just RH | |
| 12:37:46 | artom | cdent, well, with the "upstream" operators, we can just send off a thing to the mailing list. It's not perfect, but it's a start | |
| 12:37:55 | sean-k-mooney | we try to infer that based on the bugs that get reported but ya we dotn really have a good way of doing it | |
| 12:37:55 | artom | To my knowledge, no such mechanism exists for our customers | |
| 12:38:27 | cdent | one gets the impression that the number of operators actively engaged with email is ... limited | |
| 12:39:04 | sean-k-mooney | there is proably some marketing list that you can opt in to get survays and such but i doubt that apporch would get much feedback | |
| 12:39:34 | artom | I suspect the majority of operators are paying customers | |
| 12:39:43 | artom | So they expect their provider to do that work for them | |
| 12:40:23 | cdent | I'm worried in this instance that we are solving a real/legit problem in the wrong place, but I have nothing other than "feelings" to support that | |
| 12:41:02 | artom | As we all | |
| 12:41:11 | cdent | and I certainly don't have any internal customer stuff to support any of that: we continue to not expose any numa details | |
| 12:41:20 | sean-k-mooney | was tehre a partical thing that propted this topic? | |
| 12:41:22 | artom | And I suspect this isn't the first or last occurence of such problem solving | |
| 12:41:27 | artom | sean-k-mooney, https://review.opendev.org/#/c/658510/ | |
| 12:41:29 | cdent | sean-k-mooney: we're taling about can_split | |
| 12:41:59 | sean-k-mooney | of declaring that a resouce class can come form multiple rps for numa an dsuch | |
| 12:42:16 | cdent | sean-k-mooney: I'm trying to make sure that we are solving for the real problem, rather than what is possible. | |
| 12:42:35 | cdent | there's lots of useful interaction on the spec, please join in if you've got thoughts | |
| 12:43:07 | artom | We do have a thing that analyses sosreports (our "all the things" automated log reporting tool for bugs) | |
| 12:43:09 | sean-k-mooney | ill try too but it will be a while | |
| 12:43:16 | sean-k-mooney | e.g later today/tomorrow | |
| 12:43:19 | artom | We could try to pull hardware/workload info out of that | |
| 12:43:28 | sean-k-mooney | im currentlytrying to find a downstream bug fix | |
| 12:43:31 | cdent | i'm feeling a bit "sure we _can_ split, but do we want to?" | |
| 12:43:44 | artom | There's selection bias for those who actually report logs, but it's a start | |
| 12:44:06 | artom | Don't think we could get instance to host mapping info out of it, though | |
| 12:44:12 | artom | *report bugs | |
| 12:44:37 | sean-k-mooney | cdent: the only use case i have for can split i think is when i ask for more cpu then might resonable fit on one numa node so i say you can split that if that is required | |
| 12:44:53 | sean-k-mooney | but that only really makes sese for non numa guests | |
| 12:45:02 | sean-k-mooney | *sense | |
| 12:45:20 | artom | Well that was the main driver IIUC - fitting non-NUMA guests on NUMA host when every host is presenting itself as a tree | |
| 12:46:38 | artom | Anyways, need to drive kids to daycare | |
| 12:47:08 | sean-k-mooney | artom: right you only want to use can split for cpus ro memroy if you dont care about numa toplogy | |
| 12:47:57 | sean-k-mooney | all hosta are numa hosts even most high end mainstream desktop systems these days | |
| 13:02:03 | cdent | sean-k-mooney: they are, but they don't have to declare that they are | |
| 13:06:19 | sean-k-mooney | that is true. i think its clean to always declare them as such but im not oposed to not doing that | |
| 13:07:32 | sean-k-mooney | i kind of always felt that the root resouce provdier shoudl not be owned by any one service and we should therefor not have inventories in it | |
| 13:08:10 | sean-k-mooney | mainly to make it simpler for neutron cyborg and other serivce to creat resouces on the same host that are not managed by nova in plcement | |
| 13:08:29 | sean-k-mooney | but that is tangentel to split vs non split | |
| 13:20:08 | yonglihe_ | sean-k-mooney: would you like to put this on your review queue, thanks. https://review.opendev.org/#/c/658716/ | |
| 13:20:44 | sean-k-mooney | ah reporting the numa toplogy via the api sure ill take a look | |
| 13:21:09 | yonglihe_ | sean-k-mooney: thanks, looking forward to your comments! | |
| 13:42:05 | zigo | When I attempt to migrate an instance, I get this in nova-api.log: http://paste.openstack.org/show/752159/ | |
| 13:42:10 | zigo | What's going on, and how to fix? | |
| 14:02:47 | zigo | cdent: Any idea what can cause this? http://paste.openstack.org/show/752159/ | |
| 14:02:57 | cdent | zigo: looking | |
| 14:03:30 | zigo | Is Nova looking for a server group that doesn't exist? | |
| 14:06:23 | mriedem | zigo: what release? | |
| 14:06:35 | zigo | mriedem: Rocky. | |
| 14:06:44 | zigo | :P | |
| 14:06:56 | mriedem | was the server group deleted? | |
| 14:07:30 | cdent | zigo: i would have flailed | |
| 14:07:37 | zigo | mriedem: I don't think so, it just went into our availability zone called "AZ3" ... | |
| 14:08:02 | zigo | So I get "properties group='AZ3'" when doing server show. | |
| 14:10:36 | mriedem | so it's blowing up here https://github.com/openstack/nova/blob/stable/rocky/nova/objects/request_spec.py#L523 which makes me think the request spec was stored with an instance group without a uuid | |
| 14:10:40 | mriedem | is this an old instance? | |
| 14:11:05 | zigo | mriedem: No. | |
| 14:11:13 | zigo | It's a fairly recent deployment. | |
| 14:11:16 | aarents | Hello! Do you still have any concern regarding this: https://review.opendev.org/#/c/659054 can we workflow+1 ? mriedem | |
| 14:12:32 | mriedem | zigo: so you just created a server in a group and then tried to cold migrate or resize it and it blows up like this? | |
| 14:13:56 | zigo | mriedem: Someone created a server, the compute had a hardware issue, it rebooted, then I get no connectivity with that instance anymore (ovs-vsctl shows some qvo devices missing) so I'm trying to migrate it to fix the networking problem. | |
| 14:14:00 | zigo | Then I get a 500 error ... | |
| 14:14:26 | zigo | I'm trying to cold migrate indeed. | |
| 14:14:48 | zigo | And the trace is what is shown in nova-api.log when doing "nova server migrate" of that instance. | |
| 14:15:09 | mriedem | ok are you able to dump the request_specs entry for that instance? it should be a json blob. | |
| 14:15:25 | zigo | mriedem: How do I do that? | |
| 14:15:31 | zigo | You mean debugging ? | |
| 14:15:45 | zigo | Or just looking in the db? | |
| 14:15:50 | mriedem | select spec from nova_api.request_specs where instance_uuid=<instance id>; | |
| 14:18:40 | zigo | mriedem: http://paste.openstack.org/show/752161/ | |
| 14:19:35 | mriedem | i could see where we could hit an issue by setting an instance_group on the request spec without a uuid here https://github.com/openstack/nova/blob/stable/rocky/nova/objects/request_spec.py#L228 but that is meant to be some legacy code | |