Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-03-27
15:41:46 artom Could we do that in the scheduler?
15:41:48 mriedem artom: yes, see that patch i linked
15:41:51 bauzas but that's an implementation detail, IHMO
15:41:58 artom So that straight away we have a dest that supports it?
15:42:07 mriedem artom: we could if we exposed the capability as a trait on the compute and added a pre-request filter
15:42:19 bauzas artom: the scheduler doesn't know whether it's a migration or a boot
15:42:26 bauzas but what mriedem wrote
15:42:34 mriedem artom: that would be an optimization imo
15:42:40 mriedem worth noting in the spec though
15:42:41 bauzas zactly
15:42:45 openstackgerrit Chris Dent proposed openstack/nova master: [placement] Filter allocation candidates by forbidden traits in db https://review.openstack.org/556660
15:42:45 openstackgerrit Chris Dent proposed openstack/nova master: [placement] Filter resource providers by forbidden traits in db https://review.openstack.org/556472
15:42:46 openstackgerrit Chris Dent proposed openstack/nova master: [placement] Support forbidden traits in API https://review.openstack.org/556820
15:42:46 openstackgerrit Chris Dent proposed openstack/nova master: [placement] Parse forbidden traits in query strings https://review.openstack.org/556819
15:42:54 artom Mmmm, forbidden traits
15:42:58 artom /homer
15:42:59 gibi mriedem: I will be available during the notification subteam meeting timeslot or before that but I have nothing to really talk about so we can simply skip the meeting if you agree
15:43:11 SamYaple so i just ran into an annoying time consuming issue. i was going an upgrade and my nova-osapi_compute service changed to reporting its hostname to the database. this created new nova-osapi_compute services in the services table. the old service entires had version 9 and this was causing all GETs to instances to fail with instance not found
15:43:20 SamYaple code in question https://github.com/openstack/nova/blob/ed55dcad83d5db2fa7e43fc3d5465df1550b554c/nova/compute/api.py#L2268
15:43:31 SamYaple no logs anywhere describing the issue :/
15:43:52 mriedem gibi: yes let's skip
15:44:01 SamYaple would a check for old osapi_compute service versions be appropriate to add to `nova-status upgrade check` ?
15:44:12 gibi mriedem: ack, let's focuse on spec reviews
15:44:17 artom mriedem, bauzas, the optimization would be the scheduler doing the version checking? And for now we let the conductor do it?
15:44:52 bauzas jaypipes: should we also discuss in https://review.openstack.org/#/c/556873/2/specs/rocky/approved/nested-resource-providers-allocation-candidates.rst about how the scheduler would pass the root RP ?
15:45:10 bauzas jaypipes: because atm, it just says "which compute node is having that UUID ?"
15:45:15 openstackgerrit Chris Dent proposed openstack/nova master: Isolate placement database config https://review.openstack.org/541435
15:45:15 openstackgerrit Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766
15:45:16 openstackgerrit Chris Dent proposed openstack/nova master: WIP: Ensure that os-traits sync is attempted only at start of process https://review.openstack.org/553857
15:45:17 jaypipes gibi: I'm happy to speak with mlavalle or anyone else about this. I just feel like the network-bandwidth spec has gotten way over-engineered.
15:45:34 mriedem SamYaple: a bit confused,
15:45:41 mriedem you said it started creating service entries in the db,
15:45:46 mriedem but that there were also old entries?
15:46:13 mriedem SamYaple: also, this sounds vaguely familiar to https://review.openstack.org/#/c/556670/
15:46:18 gibi jaypipes: I understand that it is a long spec and this part of it seems minor
15:46:20 mlavalle jaypipes, gibi: I am planning to go over the spec today
15:46:52 openstackgerrit Stephen Finucane proposed openstack/nova master: tox: Make everything work with Python 3 https://review.openstack.org/556894
15:46:52 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove 'tools/releasenotes_tox.sh' https://review.openstack.org/534383
15:46:52 openstackgerrit Stephen Finucane proposed openstack/nova master: tox: Speed things up and document them https://review.openstack.org/534382
15:46:56 jaypipes bauzas: the scheduler simply does a ComputeNode.get_all_by_uuid(), passing all the UUIDs of all providers it sees in the provider_summaries section of the allocation_candidates response. Nothing about that will be changing.
15:47:11 mlavalle I also have the impression that we can / should simplify a bit, jaypipes
15:47:15 mriedem SamYaple: at this point, if not CONF.cells.enable and we're in here https://github.com/openstack/nova/blob/ed55dcad83d5db2fa7e43fc3d5465df1550b554c/nova/compute/api.py#L2268 we likely should be puking out a warning
15:47:16 jaypipes gibi: no, it's not the length of the spec that I have issues with.
15:47:18 bauzas jaypipes: sec
15:47:29 jaypipes mlavalle: it's the crossing of scope boundaries that I have issue with.
15:47:35 SamYaple mriedem: http://paste.openstack.org/show/715421/
15:47:40 gibi mlavalle: cool, then could you please add your view about the backend selection in https://review.openstack.org/#/c/502306/21/specs/rocky/approved/bandwidth-resource-provider.rst@610
15:48:00 SamYaple the first three services inthat list were the originals. i deleted them and everything started working
15:48:08 bauzas jaypipes: say some child RP is accepting the query, the placement API call to allocation_candidates will return the root RP UUIDs in the provider_summaries ?
15:48:28 bauzas jaypipes: is that already the case or is that requiring some implementation change ?
15:48:35 jaypipes gibi: the thing we are attempting to consume is a chunk of network (ingress or egress) bandwidth for a physical network. I don't really see how vNIC type nor network "backend" selection is relevant to that.
15:48:37 bauzas because I want to understand what's missing
15:48:38 gibi mlavalle: if we want to have what is described in that subsection then we have to convince jaypipes to have backend specific traits in the RP tree
15:48:53 jaypipes bauzas: yes. and it already does that.
15:48:57 SamYaple mriedem: with the old services not deleted, GETs on instances (nova show's) would fail
15:48:58 bauzas \o/
15:49:12 bauzas I was out of nested RPs for a while
15:49:13 mriedem SamYaple: ok so before this, the api code just relied on getting the instances from the nova db configured in your nova-api nova.conf [database]/connection field right?
15:49:20 bauzas but that's a very nice point
15:49:20 gibi jaypipes: by claiming bandwidth we implicitly select the backend, that is why backend comes into the picture
15:49:41 mriedem SamYaple: oh are you saying the new 15-level services are fixing the problem for you then?
15:49:53 mriedem but you didn't know about those until you had figured out the problem with the older service versions
15:49:59 SamYaple yes
15:50:02 mriedem ok
15:50:26 jaypipes gibi: but why should the nova scheduler care about that? shouldn't the nova scheduler just claim resources against the provider of network bandwidth for a specific physical network and then leave it to os-vif and the compute node to pick whatever network backend it wants?
15:50:28 gibi jaypipes: we failed to find a RP model that allows claiming bandwidth but not selecting a specific backend that bandwidth belongs to
15:50:45 mriedem SamYaple: did you just upgrade to newton? or ocata/pike?
15:51:22 SamYaple mriedem: newton -> ocata
15:51:40 mriedem ok and now you have a cell1 mapping for the nova db
15:51:47 SamYaple oh yea its all working now
15:51:49 mriedem so you're pulling instances from https://github.com/openstack/nova/blob/ed55dcad83d5db2fa7e43fc3d5465df1550b554c/nova/compute/api.py#L2271
15:51:59 mriedem ok
15:52:26 mriedem at this point i'm not sure what the nova-status version check would look for, nova-osapi_compute services with version < 15 across all cells?
15:52:32 mriedem and then backport that to queens, pike and ocata?
15:52:40 gibi jaypipes: could be a problem with our model
15:52:40 SamYaple mriedem: i think so
15:52:51 SamYaple there are no logs indicating any issues
15:53:04 SamYaple mriedem: i had to walk to code and db (which im not super familiar with) to findthis
15:53:11 mriedem SamYaple: yeah like i said at this point that should be a warning, but a warning in rocky doesn't help you in ocata
15:53:22 mriedem SamYaple: can you report a bug and i can bring it up in the cells meeting?
15:53:29 SamYaple im looking to the future :)
15:53:32 SamYaple sure will do
15:53:47 mriedem whatever we do can at least help the next poor soul that runs into this
15:53:52 SamYaple exactly
15:53:55 mriedem thanks
15:54:43 SamYaple nova uses launchpad still, right?
15:54:51 mriedem hell yeah
15:55:02 SamYaple :)
15:55:41 Spaz-Home Trade me plz.
15:56:02 jaypipes gibi, mlavalle: please move convo to https://etherpad.openstack.org/p/X0RboWOe7C
15:56:08 gibi jaypipes: ack
16:05:03 openstack Launchpad bug 1759316 in OpenStack Compute (nova) "pre-cells_v2 nova-osapi_compute service in database breaks instance lookup" [Undecided,New]
16:05:03 SamYaple mriedem: https://bugs.launchpad.net/nova/+bug/1759316
16:06:24 jaypipes mlavalle: are you light green on etherpad?
16:06:30 openstackgerrit Surya Seetharaman proposed openstack/nova master: Scheduling Optimization: Remove cell0 from the list of candidates https://review.openstack.org/556821
16:06:31 openstackgerrit Surya Seetharaman proposed openstack/nova master: Add --enable and --disable options to nova-manage update_cell https://review.openstack.org/555416
16:06:31 openstackgerrit Surya Seetharaman proposed openstack/nova master: Allow scheduling only to enabled cells (Filter Scheduler) https://review.openstack.org/550527
16:06:32 openstackgerrit Surya Seetharaman proposed openstack/nova master: Update the cells FAQs and scheduler maintenance docs. https://review.openstack.org/556459
16:06:44 mlavalle jaypipes: I am green
16:07:03 jaypipes mlavalle: cool, thx :)

Earlier   Later