Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-01-14
15:27:48 belmoreira efried: hi
15:28:25 efried belmoreira: Greetings. Today in the nova-scheduler meeting we brought up a couple of patches that CERN had showed interest in.
15:28:53 efried We were looking to find out if y'all could (re)deploy them to make sure they do what's intended and don't break the world.
15:29:00 efried (jaypipes heads up)
15:29:48 efried The first one was the placement traffic improvement series starting at https://review.openstack.org/#/c/615677/
15:30:11 efried IIRC, you had deployed it and demonstrated the expected improvements, which is great.
15:30:11 efried but
15:30:22 efried there was some concern about how it would behave in large ironic deployments.
15:30:36 efried Late last week I made some updates to hopefully take care of that
15:31:25 efried Last I recall, you were about to start deploying to your ironic nodes - not sure how far you got on that or whether you stalled it because of the perceived problems with the patches.
15:31:43 belmoreira efried: that are great news
15:31:54 efried Anyway, we were hoping you could redeploy the series in its current form to see if it's copacetic
15:34:03 belmoreira before the end of the year I was trying it with the ironic compute nodes. It was in our small test cell, and it was OK. Scalability issues will only be noticed in our prod instance. Was planning to do it next week.
15:34:37 belmoreira will check the new code, thanks.
15:36:16 efried belmoreira: Great.
15:36:40 efried The other one was jaypipes: https://review.openstack.org/#/c/623558/
15:36:40 efried I don't know anything about this, will let him speak to it.
15:43:09 cfriesen artom: have you made any progress with the resource-aware live migration stuff? I don't see any reviews up since the spec review.
15:43:56 sean-k-mooney cfriesen: i haven't seen artom online downstream yet today so not sure if he is about yet
15:44:26 cfriesen sean-k-mooney: thanks
15:44:54 belmoreira efried: I didn't try jaypipes code because merge/dependencies issues. Ended up with a custom code that only gets the info from the nodes in the cell. Will revisit it next week.
15:45:06 efried ight
15:45:18 jaypipes belmoreira: https://review.openstack.org/#/c/623558/ commit message should be self-explanatory. It should dramatically increase performance of your scheduler due to not doing N queries and only doing 1 query per cell.
15:45:29 sean-k-mooney cfriesen: i know he plans to work on it this sprint and once i finsih off the last few bits for the sriov migraton ill likely try and spend my time helping him with that work
15:46:28 cfriesen sean-k-mooney: good to hear. I'm hoping to have some review cycles for this.
15:52:05 belmoreira jaypipes: let me revisit again your patch. I wasn't able to use your code directly because it was relying in few things that are new to rocky and I got scared to push them to prod. But we cooked a light similar thing. This is the result in the schedule time: https://docs.google.com/document/d/1atA0wbUNBCdojiq8WyBKA0k8Ulyj91bzXDMPOIN9gRI
15:52:24 belmoreira jaypipes: I think yours will improve it even more
15:55:07 belmoreira jaypipes: after this what we observed is that now most of the time is because the placement call... But needs more investigation in our side.
15:56:53 belmoreira jaypipes: will let you know when we start again our tests. Thanks for all this work
15:57:55 jaypipes belmoreira: rock on :)
16:11:30 adrianc sean-k-mooney: working on the nits, PS will be uploaded by tommorow
16:12:38 adrianc sean-k-mooney: left to deal with several on the indirect implementation commit and ill upload
16:13:05 sean-k-mooney ok ill wait untill you have uploaded it so to rebase
16:13:25 artom sean-k-mooney, I'm around
16:13:33 artom cfriesen, nothing posted yet, working on it
16:13:35 adrianc sean-k-mooney: thanks
16:13:55 artom sean-k-mooney, just because I don't say anything, doesn't mean I'm not here ;)
16:13:59 cfriesen artom: cool, thanks.
16:21:29 sean-k-mooney artom: ah lurking in the backgound. sneaky :)
16:23:21 artom sean-k-mooney, I mean, it's sort of what everyone does by default, no?
16:26:10 sean-k-mooney artom: ya true but i usually say hi in the morning
16:27:51 artom sean-k-mooney, maybe I'll start doing that :)
16:29:39 openstackgerrit Balazs Gibizer proposed openstack/nova master: Reject interface attach with QoS aware port https://review.openstack.org/570078
16:29:40 openstackgerrit Balazs Gibizer proposed openstack/nova master: Reject networks with QoS policy https://review.openstack.org/570079
16:29:40 openstackgerrit Balazs Gibizer proposed openstack/nova master: Create RequestGroup from neutron port https://review.openstack.org/625941
16:29:41 openstackgerrit Balazs Gibizer proposed openstack/nova master: Include requested_resources to allocation candidate query https://review.openstack.org/625942
16:29:43 openstackgerrit Balazs Gibizer proposed openstack/nova master: Transfer port.resource_request to the scheduler https://review.openstack.org/567268
16:29:44 openstackgerrit Balazs Gibizer proposed openstack/nova master: Extend RequestGroup object for mapping https://review.openstack.org/619527
16:29:44 openstackgerrit Balazs Gibizer proposed openstack/nova master: Calculate RequestGroup resource provider mapping https://review.openstack.org/616239
16:29:45 openstackgerrit Balazs Gibizer proposed openstack/nova master: Fill the RequestGroup mapping during schedule https://review.openstack.org/619528
16:29:45 openstackgerrit Balazs Gibizer proposed openstack/nova master: Pass resource provider mapping to neutronv2 api https://review.openstack.org/616240
16:29:46 openstackgerrit Balazs Gibizer proposed openstack/nova master: Recalculate request group - RP mapping during re-schedule https://review.openstack.org/619529
16:29:49 openstackgerrit Balazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding https://review.openstack.org/569459
16:29:50 openstackgerrit Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317
16:29:51 openstackgerrit Balazs Gibizer proposed openstack/nova master: Remove port allocation during detach https://review.openstack.org/622421
16:29:52 openstackgerrit Balazs Gibizer proposed openstack/nova master: Refactor PortResourceRequestBasedSchedulingTestBase https://review.openstack.org/624080
16:29:52 openstackgerrit Balazs Gibizer proposed openstack/nova master: Record requester in the InstancePCIRequest https://review.openstack.org/625310
16:29:53 openstackgerrit Balazs Gibizer proposed openstack/nova master: Add pf_interface_name tag to passthrough_whitelist https://review.openstack.org/625311
16:29:54 openstackgerrit Balazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF https://review.openstack.org/623543
16:29:54 openstackgerrit Balazs Gibizer proposed openstack/nova master: Extend NeutronFixture to return port with resource request https://review.openstack.org/630719
16:29:55 openstackgerrit Balazs Gibizer proposed openstack/nova master: Read port resource request from Neutron https://review.openstack.org/630720
16:29:55 openstackgerrit Balazs Gibizer proposed openstack/nova master: Reject server create with port having resource request https://review.openstack.org/630721
16:29:56 openstackgerrit Balazs Gibizer proposed openstack/nova master: Reject resize with port having resource request https://review.openstack.org/630722
16:29:56 openstackgerrit Balazs Gibizer proposed openstack/nova master: Reject migrate with port having resource request https://review.openstack.org/630723
16:29:57 openstackgerrit Balazs Gibizer proposed openstack/nova master: Reject evacuate with port having resource request https://review.openstack.org/630724
16:29:57 openstackgerrit Balazs Gibizer proposed openstack/nova master: Reject unshelve with port having resource request https://review.openstack.org/630725
16:31:07 sean-k-mooney hehe i takeit you have resovled all the merge conflits
16:32:18 gibi sean-k-mooney: not just that, but make it so that the new external behavior is only active in a new microversion (which still needs to be added on the top)
16:32:35 gibi mriedem: ^^ fyi
16:33:07 sean-k-mooney gibi: cool and for the old micro version dose it ignore the request or reject
16:33:36 gibi sean-k-mooney: today it rejects. Ignoring the request leads to some hard to manage edge cases that I added to the etherpad
16:34:02 gibi sean-k-mooney: like create a server with a new microversion so there is port allocation but then migrate it with an old microversion
16:34:49 sean-k-mooney ok well we can disucss those on the call i guess
16:34:59 gibi sean-k-mooney: yes, definitely
16:35:11 sean-k-mooney on blance that might be safer but we need a failly big upgrade impact section to call it out
16:35:24 sean-k-mooney did you choose a time by the way
16:35:36 gibi sean-k-mooney: I still have to talk to mlavalle about it
16:35:56 gibi sean-k-mooney: as the last proposed time, Tuesday, does not seem to work any more
16:36:19 mlavalle gibi: will get back to you in ~30 mins
16:36:27 gibi mlavalle: OK, no worries
16:36:36 sean-k-mooney ok cool ill try and be there but i also dont need to be so ill keep an eye out on the ml
16:37:04 gibi sean-k-mooney: i will post to the ML in any case
16:38:09 gibi sean-k-mooney: btw if we go with the reject solution in nova, we can still turn off sending the resource request in neutron. That way we can get back the true legacy behavior, i.e. no resource request in the port et all
16:39:41 sean-k-mooney gibi: im not quite sure how you neutron would know when to do that. are you suggesting a config opion for it?
16:40:19 gibi sean-k-mooney: yes, it is config driven in this WIP patch https://review.openstack.org/#/c/627978/
16:40:58 gibi sean-k-mooney: it turned out that neutron api extensions are not selectively loaded but loaded by the service plugin
16:41:22 gibi sean-k-mooney: but everything in the neutron code is ready to make an api extension like our resource request selectively loaded
16:42:01 sean-k-mooney ah cool. so if you have stien neutron and rocky nova or older you would set the config option to disable it
16:42:48 sean-k-mooney gibi: well there are selectivly loaded in that you have to enable the service plugin or extentinon in the neutron.conf or in the agent/ml2_conf.ini
16:43:04 gibi sean-k-mooney: if you have stein neutron and rocky nova then nova does not check the resource request at all. if you have stein neutron and stein nova, but you want to go back to the legacy behavior then you can turn off the extension in neutron
16:43:31 gibi sean-k-mooney: as far as I know if you enable a service plugin then it loads every extension of that plugin
16:44:47 sean-k-mooney ok so its not that granular in say enable ingress qos only you can either enable qos or not
16:45:02 gibi sean-k-mooney: yes, as far as I understand
16:45:28 sean-k-mooney and you get all the qos oplicys or none today. i honestly did not look quite that closely at least not in a few years
16:47:03 gibi sean-k-mooney: I've never looked into it deeply. I rely on rubasov's and lajoskatona's neutron experties
16:48:07 ade_lee hi all -- we have a grenade test that keeps failing due to an error which is related to volume attachment. Is there anyone here that knows about how nova volume attach works and can help?
16:49:33 jangutter moshel, lennyb: I suspect the OVS_HW_offload CI is having some issues. Would you be able to check it?
16:50:48 ade_lee specifically, we're running into an error where we try to write to a volume before its ready --
16:51:33 ade_lee we do wait for cinder to show the volume as being "in-use", but that seems to be insufficient.
16:51:59 ade_lee what would we wait for to ensure that the volume is ready for use?

Earlier   Later