Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-07-20
10:36:32 gibi I can try to put together a test for this scenario
10:36:58 gibi but this time the problem is not on the placement side but on the scheduler/manager side
10:38:32 cdent gibi: I reckon a bug report describing what you just said is sufficient
10:39:21 cdent “I consumed all my MAGIC” would make a good t-shirt or epitath
10:40:00 gibi OK I file a bug
10:41:19 cdent I’ve added a link to this talk to the review, for visibility
10:41:20 gibi and I made a note about the t-shirt
10:42:49 cdent I’m _really_ glad you are tracking this stuff so closely. I’d been planning to, but haven’t had the time. I think there are going to be lots of edge cases and other weirdnesses. Which is all fine. We’ll make it happy soon enough.
10:43:26 openstackgerrit Stephen Finucane proposed openstack/os-vif master: doc: Remove cruft from releasenotes conf.py https://review.openstack.org/480092
10:44:59 abhishekk hi can any one confirm that uwsgi will only be used for development purpose or it can be used in production as well?
10:47:45 gibi bauzas, cdent: bug has been reported https://bugs.launchpad.net/nova/+bug/1705446
10:47:47 openstack Launchpad bug 1705446 in OpenStack Compute (nova) "filter scheduler raises TypeError: argument of type 'NoneType' is not iterable when placement returns no allocation candidates" [Undecided,New]
10:48:02 cdent excellent thanks
10:48:33 stephenfin abhishekk: I can't confirm it, but I'm pretty sure any WSGI server can be used, yes
10:48:43 gibi cdent: yes, we just figure out that we might need to use the custom resource feature to build up a solution so I started building a PoC out of what nova has
10:49:27 abhishekk stephenfin: ok, thank you
10:49:52 cdent abhishekk: the idea with the new wsgi script is to make it so it can be run with any wsgi server
10:50:37 cdent gibi: I’ve been hearing that quite a lot: custom resource classes are going to allow a lot of interesting flexibility for other hypervisors
10:51:06 abhishekk cdent: ok, got it, thank you :)
10:51:19 gibi cdent: I think it will be a buzzword soon in the telco world
10:51:58 cdent gibi: I hope we don’t run into too many people who are trying to do what nested resource providers is supposed to provide, with custom resource classes instead
10:52:36 cdent gibi: it won’t be the end of the world or anything, but it will be bad for interop
10:52:49 cdent (which I guess in the telco world is not a huge concern, but still)
10:52:54 openstackgerrit Merged openstack/nova master: Handle None returned from get_allocation_candidates due to connect failure https://review.openstack.org/484988
10:54:23 gibi cdent: does nested resources support different provider for the parent and the child resource?
10:55:25 gibi cdent: I mean my MAGIC is provided by the compute node so in theory this can be nested under the compute resource somehow but the logic of providing MAGIC does not fitt nova so I need a separate provider
10:59:19 openstackgerrit Dao Cong Tien proposed openstack/nova master: Fix invalid getattr/setattr for DriverBlockDevice https://review.openstack.org/485550
11:00:01 openstackgerrit Merged openstack/nova master: add dict of allocation requests to select_dests() https://review.openstack.org/483565
11:01:36 cdent gibi: yeah, one example is compute node provider -> pci devices that provide physical or virtual functions (and have N of them)
11:01:46 cdent and mix numa cells into that
11:03:04 gibi cdent: sounds cool.
11:03:36 cdent it is, but you know how the sql is already sort of mind blowing? now add a graph of providers ot that...
11:04:08 openstackgerrit Stephen Finucane proposed openstack/nova master: remove un-necessary update() in _init_compute_node of rt https://review.openstack.org/483506
11:06:25 openstackgerrit Stephen Finucane proposed openstack/nova master: objects: remove cpuset_reserved from __init__ of InstanceNUMATopology https://review.openstack.org/466030
11:06:25 openstackgerrit Stephen Finucane proposed openstack/nova master: objects: remove pagesize from __init__ of InstanceNUMATopology https://review.openstack.org/485553
11:06:26 openstackgerrit Stephen Finucane proposed openstack/nova master: objects: remove related pinning from __init__ of InstanceNUMATopology https://review.openstack.org/485554
11:18:07 openstackgerrit Andrey Volkov proposed openstack/nova master: Change default policy to view quota details https://review.openstack.org/386008
11:19:35 gibi cdent: that SQL is something I cannot troubleshoot with my current SQL experiences so we need a cloning machine to replicate Jay
11:20:37 cdent it sort of makes sense when it is broken down into pieces, but it can be a bit difficult to see that, and then the ways in which those pieces assemble can be a bit weird, as that bug you found shows
11:22:51 openstackgerrit Chris Dent proposed openstack/nova master: remove un-necessary update() in _init_compute_node of rt https://review.openstack.org/483506
11:28:30 openstackgerrit Chris Dent proposed openstack/nova master: [placement] fix rp create failed when late placement endpoint https://review.openstack.org/483460
11:37:51 gibi cdent:
11:38:14 gibi cdent: I think I can fix the bug by moving the proper initialization from here https://review.openstack.org/#/c/483565/4/nova/scheduler/manager.py@139
11:38:30 gibi cdent: to here https://review.openstack.org/#/c/483565/4/nova/scheduler/manager.py@113
11:42:01 cdent that would address your particular issue, but my understanding is that “None” is being used in some contexts (the caching scheduler?) to mean something. None is sometimes expected in the _schedule method of filter_scheduler
11:42:37 openstackgerrit Takashi NATSUME proposed openstack/nova master: List/show all server migration types (1/2) https://review.openstack.org/430608
11:44:34 cdent gibi: I’ve got a different batch of code checked out at the moment, digging around in another bug, so I’ve not got the right mental state for this but will in a little while
11:45:06 openstackgerrit Alexandru Muresan proposed openstack/nova master: Pass config object to oslo_reports https://review.openstack.org/485575
11:45:38 openstackgerrit Radoslav Gerganov proposed openstack/nova master: VMware: serial console log (completed) https://review.openstack.org/450636
11:46:04 gibi cdent: sure. I will try to propose something as a patch in the meantime
11:48:20 openstackgerrit Chris Dent proposed openstack/nova master: placement: proper JOIN order for shared resources https://review.openstack.org/485088
11:48:21 openstackgerrit Chris Dent proposed openstack/nova master: [placement] gabbi tests for shared custom resource class https://review.openstack.org/485209
11:48:22 openstackgerrit Chris Dent proposed openstack/nova master: placement: alloc candidates only shared resources https://review.openstack.org/484900
11:48:26 cdent dammit
11:49:05 cdent that wasn’t supposed to rebase everything
11:49:33 openstackgerrit OpenStack Proposal Bot proposed openstack/nova master: Imported Translations from Zanata https://review.openstack.org/477091
11:52:04 mkucia heh, unintended rebases happen time to time :)
12:03:44 openstackgerrit Balazs Gibizer proposed openstack/nova master: placement: init alloc_reqs earlier https://review.openstack.org/485585
12:04:47 gibi cdent: this ^^ is the patch I put together
12:05:05 cdent cool, wil look in a sec
12:05:34 gibi cdent: thanks
12:12:41 openstackgerrit Merged openstack/nova master: Log compute node uuid when the record is created https://review.openstack.org/484447
12:13:26 openstackgerrit Balazs Gibizer proposed openstack/nova master: placement: init alloc_reqs earlier https://review.openstack.org/485585
12:16:05 openstackgerrit Alex Szarka proposed openstack/nova master: Add method for verify multiple versioned notifications https://review.openstack.org/465526
12:20:50 gibi cdent: thanks for the comment on the patch. Let's wait for the others to wake up to discuss if this is the good fix for the problem
12:21:18 cdent yeah, that’s pretty much what I was trying to say in my comment, in too many words
12:21:29 gibi cdent: then I got your message :)
12:22:16 openstackgerrit Chris Dent proposed openstack/nova master: Add functional test for local delete allocations https://review.openstack.org/470578
12:22:32 openstackgerrit Alex Szarka proposed openstack/nova master: Raise Exception instead of Exception method call https://review.openstack.org/482200
12:26:14 openstackgerrit Merged openstack/nova master: Fix indentation in policy doc https://review.openstack.org/484646
12:26:21 openstackgerrit Balazs Gibizer proposed openstack/nova master: placement: init alloc_reqs earlier https://review.openstack.org/485585
12:30:12 openstackgerrit Merged openstack/nova master: Only setup iptables for metadata if using nova-net https://review.openstack.org/480765
12:31:11 openstackgerrit Merged openstack/nova master: Add log info in scheduler to mark start of scheduling https://review.openstack.org/481340
12:32:01 openstackgerrit Merged openstack/nova master: Updated from global requirements https://review.openstack.org/485410
12:32:50 openstackgerrit Merged openstack/nova master: VStorage: changed default log path https://review.openstack.org/458557
12:33:51 openstackgerrit Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766
12:49:04 openstackgerrit Takashi NATSUME proposed openstack/nova master: List/show all server migration types (2/2) https://review.openstack.org/459483
12:50:20 openstackgerrit Matthew Booth proposed openstack/nova master: Allow wrapping of closures https://review.openstack.org/479801
12:50:21 openstackgerrit Matthew Booth proposed openstack/nova master: Use _error_out_instance_on_exception in finish_resize https://review.openstack.org/485601
12:51:37 openstackgerrit Sean Dague proposed openstack/nova master: WIP: request_log addition for running under uwsgi https://review.openstack.org/485602
13:01:04 cdent sdague: I wrote a couple of alternative ideas (to avoid paste changes) on ^. If it seems useful and you haven’t got the time I can probably look into tomorrow.
13:08:13 mdbooth dansmith: https://review.openstack.org/#/c/479802/
13:08:56 mdbooth You wanted me to break it up, but as I put in a comment I don't think that works.
13:10:44 cdent gibi: if you learn something while I’m away, let me know
13:11:39 mdbooth dansmith: Anyway, working on it now so I'd really like to make sure I've understood you.
13:11:58 openstackgerrit iswarya vakati proposed openstack/nova master: Fixed wrap from taking negative values https://review.openstack.org/481465
13:13:42 dansmith mdavidson: what doesn't work?
13:14:09 dansmith heh, mdbooth ^
13:14:14 bauzas gibi: so, could you please tl:dr the problem about https://bugs.launchpad.net/nova/+bug/1705446 ?
13:14:15 openstack Launchpad bug 1705446 in OpenStack Compute (nova) "filter scheduler raises TypeError: argument of type 'NoneType' is not iterable when placement returns no allocation candidates" [High,In progress] - Assigned to Balazs Gibizer (balazs-gibizer)
13:16:49 openstackgerrit Alex Szarka proposed openstack/nova master: Transform the transformed notifications functional tests https://review.openstack.org/483448
13:16:49 gibi bauzas: hi! so when placement returns no allocation candidates because there is no resource left, the scheduler/manager calls the scheduler drivers with None in alloc_reqs_by_rp_uuid
13:17:02 gibi bauzas: then the FilterScheduler tries to iterate on alloc_reqs_by_rp_uuid and blows up
13:17:29 gibi bauzas: see also https://review.openstack.org/#/c/485585/3/nova/scheduler/manager.py
13:18:17 gibi bauzas: alloc_reqs_by_rp_uuid onyl properly initialized in the else branch at line 147
13:18:31 mdbooth dansmith: I think it's an atomic change
13:18:41 openstackgerrit Alex Szarka proposed openstack/nova master: Transform instance.exists notification https://review.openstack.org/403660
13:18:54 mdbooth dansmith: I appreciate it's obtuse
13:19:14 gibi bauzas: as soon as I moved the initialization up to the top the FilterScheduler was happy again
13:19:21 mdbooth But being obtuse is the problem

Earlier   Later