| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-12-19 | |||
| 17:26:10 | stephenfin | Bhujay: What is going on with your IRC connection? | |
| 17:27:25 | openstackgerrit | Merged openstack/nova master: Address nits on I1f1fa1d0f79bec5a4101e03bc2d43ba581dd35a0 https://review.openstack.org/614323 | |
| 17:27:32 | openstackgerrit | Merged openstack/nova master: Fix a broken-link in nova doc https://review.openstack.org/626113 | |
| 17:32:50 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/ocata: Handle binding_failed vif plug errors on compute restart https://review.openstack.org/626369 | |
| 17:32:52 | mriedem | hooray for ocata em ^ | |
| 17:34:39 | stephenfin | melwitt: Seeing as you looked at the earlier change, fancy taking a look at https://review.openstack.org/#/c/626228 ? | |
| 17:35:18 | melwitt | stephenfin: sure, always up for being pinged for reviews | |
| 17:35:30 | stephenfin | mriedem: Thanks for reviewing that nit patch (y) | |
| 17:36:16 | mriedem | the docs one? i didn't really, just saw gibi was +2 and it was a rebase | |
| 17:36:24 | mriedem | but yw :) | |
| 17:39:31 | openstackgerrit | Jack Ding proposed openstack/nova-specs master: Select cpu model from a list of cpu models https://review.openstack.org/620959 | |
| 17:57:02 | openstackgerrit | Chris Dent proposed openstack/nova master: Redirect user/placement to placement docs https://review.openstack.org/626333 | |
| 18:02:02 | openstackgerrit | Krzysztof Opasiak proposed openstack/nova master: Fix server IPs with non-unique network names https://review.openstack.org/625371 | |
| 18:02:15 | cfriesen | stephenfin: any chance you could take a look at the cpu models spec proposed by Jack ^ ? Basically instead of setting one model in nova.conf the operator could specify a list, and the virt driver would use the first one that provides the requested cpu features. | |
| 18:02:58 | openstackgerrit | Krzysztof Opasiak proposed openstack/nova master: Fix server IPs with non-unique network names https://review.openstack.org/625371 | |
| 18:16:00 | melwitt | mriedem: I wanted to bring this review to your attention, bug about returning build requests when a marker is specified (I know you love paginating stuff). I'm +2 on it https://review.openstack.org/624870 | |
| 18:22:56 | openstackgerrit | Merged openstack/nova master: Remove legacy RequestSpec compat code from live migrate task https://review.openstack.org/625705 | |
| 18:39:52 | openstackgerrit | Tim Rozet proposed openstack/nova master: Fixes race condition with privsep utime https://review.openstack.org/625741 | |
| 18:57:55 | mriedem | i saw it before, asked andrey to flesh out the commit message, haven't been back | |
| 19:16:41 | melwitt | ah, ok | |
| 19:28:05 | mnaser | friendly bump on this - https://review.openstack.org/#/c/619352/ | |
| 19:28:21 | mnaser | simple backport, the changes in the newer branches have merged too | |
| 19:29:10 | mriedem | frickler: fyi redo of the queens release https://review.openstack.org/626377 | |
| 19:31:22 | mriedem | duplicate bug of https://review.openstack.org/#/c/567701/ just came through triage, the fix is straight-forward, the patch is mostly a functional test | |
| 19:58:40 | openstackgerrit | sean mooney proposed openstack/nova master: PCI: do not force remove allcoated devices https://review.openstack.org/626381 | |
| 19:59:20 | sean-k-mooney | mriedem: i have no idea why my unit test is not working in ^ | |
| 20:00:56 | sean-k-mooney | im going to grab dinner but if you have any insight let me know. | |
| 20:01:01 | mriedem | ack thanks | |
| 20:33:20 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Document using service user tokens for long running operations https://review.openstack.org/626388 | |
| 20:35:00 | openstackgerrit | Jack Ding proposed openstack/nova-specs master: Select cpu model from a list of cpu models https://review.openstack.org/620959 | |
| 20:39:51 | melwitt | mriedem: re: the ML thread about that, I thought the oslo.messaging heart beat would take care of the long running live migration problem? | |
| 20:40:00 | mriedem | the problem isn't rpc | |
| 20:40:21 | melwitt | oh, the token auth expiring | |
| 20:40:24 | mriedem | nova tries to make a rest api request using the users token to cinder, | |
| 20:40:26 | mriedem | the token has timed out | |
| 20:40:31 | melwitt | I see, ok | |
| 20:41:06 | melwitt | yeah, have to have both then. I got the two confused together but they are two different issues | |
| 20:41:17 | mriedem | i want to say i heard anecdotes at one point that rax public cloud had 24 token timeouts because of stuff like this way back when | |
| 20:41:58 | mriedem | the service user token stuff was added by osic, which was rax+intel | |
| 20:43:55 | melwitt | yeah, sounds familiar. I feel like we had something similar at yahoo too | |
| 20:44:21 | melwitt | *something similar to service user auth | |
| 20:58:21 | mriedem | long_rpc_timeout probably also deserves some mention somewhere in troubleshooting admin docs, but not sure right now, | |
| 20:58:34 | mriedem | in general i've had random thoughts about things that would be good to put into a 'scaling issues' page in the docs | |
| 20:58:39 | openstackgerrit | Krzysztof Opasiak proposed openstack/nova master: Fix server IPs with non-unique network names https://review.openstack.org/625371 | |
| 20:58:40 | mriedem | but haven't started anything | |
| 20:59:18 | melwitt | ++ | |
| 21:15:08 | mriedem | melwitt: i'm going to fix that unnecessary for loop in https://review.openstack.org/#/c/624870/ that Kevin pointd out, then approve | |
| 21:15:46 | melwitt | ok, sounds good | |
| 21:30:17 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Exclude build request marker from server listing https://review.openstack.org/624870 | |
| 21:41:17 | melwitt | mriedem: so the func test in this change doesn't fail without the change https://review.openstack.org/567701 is that expected based on the commit message? if so, is there no way to demonstrate the bug in the test? | |
| 21:42:11 | melwitt | I wasn't sure based on the wording "that is not a regression" | |
| 21:43:05 | mriedem | been awhile, but the commit message is saying I8d426f2635232ffc4b510548a905794ca88d7f99 didn't introduce a regression | |
| 21:43:51 | melwitt | ok, so unrelated to what I'm seeing I think. basically, without the change, somehow AZ is being updated on the instance. I don't yet know how | |
| 21:44:40 | mriedem | i'll have to poke at it, i wrote that in may | |
| 22:00:33 | melwitt | looking at it myself for curiosity, I'm not finding how AZ could be updated without the fix. weird | |
| 22:05:24 | openstackgerrit | Merged openstack/nova master: Move a generic bridge helper to a linux_net privsep file. https://review.openstack.org/620010 | |
| 22:15:03 | mriedem | oh yay, for a looong time we passed potentially the wrong image to move claim during a resize https://github.com/openstack/nova/blob/1249617bdfaa8f4c586159374a4a0b244bbb298a/nova/conductor/tasks/migrate.py#L77 | |
| 22:15:19 | mriedem | based on the original image used to create the server, but potentially not the last image used to rebuild the server | |
| 22:15:24 | mriedem | gd req spec | |
| 22:17:26 | mriedem | which wasn't fixed until https://github.com/openstack/nova/commit/984dd8ad6add4523d93c7ce5a666a32233e02e34 inadvertently | |
| 22:17:54 | melwitt | hoo boy | |
| 22:18:18 | melwitt | me too :( | |
| 22:19:30 | mriedem | oh this also likely means that if you shelve, unshelve, resize, we're passing the original image used to create the server, not the current image meta (in case that changed) | |
| 22:23:56 | melwitt | so the AZ is still the original all the way to the end of the live migration. so how is the servers.get API returning the new AZ... the search continues | |
| 22:25:57 | mriedem | i think i know | |
| 22:26:07 | mriedem | the api code looks up the az from the instance.host | |
| 22:26:12 | mriedem | i think | |
| 22:26:21 | melwitt | pulled from the DB, the AZ is still the original in instance.availability_zone | |
| 22:26:26 | melwitt | ahhhh | |
| 22:26:40 | melwitt | must be, it can't be looking at instance.availability_zone | |
| 22:26:42 | mriedem | there are other bugs for that behavior | |
| 22:26:50 | melwitt | THANKS API | |
| 22:26:55 | mriedem | https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/views/servers.py#L185 | |
| 22:27:16 | melwitt | le sigh | |
| 22:27:18 | mriedem | https://github.com/openstack/nova/blob/master/nova/availability_zones.py#L165 | |
| 22:27:33 | mriedem | yeah if instance.host is not None, that code gets the az from the host aggregate, not the instance.az | |
| 22:27:59 | mriedem | ala https://review.openstack.org/#/c/582342/ | |
| 22:28:00 | melwitt | well, that explains it | |
| 22:28:08 | mriedem | i have a f'ing patch for everything | |
| 22:28:22 | melwitt | it's true | |
| 22:28:30 | mriedem | https://bugs.launchpad.net/nova/+bug/1782539 | |
| 22:28:30 | openstack | Launchpad bug 1782539 in OpenStack Compute (nova) "Fail to filter the list of instances by the available zone" [Medium,In progress] - Assigned to huanhongda (hongda) | |
| 22:33:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Pass request_spec from compute to cell conductor on reschedule https://review.openstack.org/582417 | |
| 22:33:11 | mriedem | speak-o-the-turd | |
| 22:34:19 | mriedem | melwitt: so i'll adjust that test to assert based on the db rather than the api | |
| 22:34:51 | melwitt | ok, makes sense | |
| 22:37:17 | melwitt | now, to finish reviewing cross-cell-resize | |
| 22:58:14 | openstackgerrit | Hongbin Lu proposed openstack/nova-specs master: [WIP] Support scheduling VM's NICs to different PFs https://review.openstack.org/626055 | |
| 23:03:11 | tonyb | I'm seeing something 'strange' where placement is always selecting the same host out of the $n available I *suspect* there is cruft in the DB after a bunch of failed boots but I don't really know | |
| 23:03:28 | tonyb | asside form the scheduler and placement logs where should I look? | |
| 23:04:25 | melwitt | tonyb: in the past, default scheduler behavior was to pack instances, so as long as a host has capacity, it would be returned again. I'm not sure if that's still the case now though | |
| 23:04:35 | melwitt | is that what you're seeing or no? | |
| 23:05:05 | mriedem | tonyb: there was a thread in the ops list about this recently, there is an option you can set to randomize the results from placement | |
| 23:05:16 | mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#placement.randomize_allocation_candidates | |
| 23:05:32 | tonyb | melwitt: Ahh perhaps that's it | |
| 23:05:58 | mriedem | ^ goes in the placement config btw, not nova-scheduler | |
| 23:06:59 | tonyb | mriedem: Okay. I should have said this is queens but I'll look for packing and/or randomising | |
| 23:07:02 | tonyb | Thanks | |
| 23:07:47 | mriedem | looky here | |
| 23:07:48 | mriedem | https://docs.openstack.org/nova/queens/configuration/config.html#placement.randomize_allocation_candidates | |