| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-10-25 | |||
| 15:05:06 | mriedem | but i'd start there rather than here | |
| 15:05:29 | gregwork | ok thanks for the contact info | |
| 15:19:44 | openstackgerrit | Eric Fried proposed openstack/python-novaclient master: Stop supporting and testing python2 https://review.opendev.org/691365 | |
| 15:23:38 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Allow evacuating server with port resource request https://review.opendev.org/688387 | |
| 15:24:45 | gibi | mriedem: one up ^^ one more to go | |
| 15:32:20 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Enable evacuation with qos ports https://review.opendev.org/688688 | |
| 15:32:48 | gibi | mriedem: done as well ^^ | |
| 15:34:00 | openstackgerrit | John Garbutt proposed openstack/nova-specs master: Add Unified Limits Spec https://review.opendev.org/602201 | |
| 15:34:03 | gibi | this marks the end of my work week. have a nice weekend folks! | |
| 15:37:03 | dansmith | gibi: wait | |
| 15:37:05 | dansmith | WAIT | |
| 15:37:45 | dansmith | gibi: in functional, I'm doing wait_for_versioned_notification() by the things it returns have empty payloads.. do I need to enable some config to have that work? | |
| 15:40:09 | dansmith | mriedem: do you know? | |
| 15:41:25 | mriedem | no, you just need to stub the fake notifier stuff | |
| 15:41:46 | dansmith | okay, I got that of course and am getting the notifs but payloads are empty | |
| 15:41:50 | dansmith | maybe I'm emitting wrong | |
| 15:46:22 | dansmith | must be because the other ones have payloads | |
| 15:46:23 | dansmith | danigt | |
| 15:48:07 | dansmith | helps if I register the payload too :D | |
| 15:48:47 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Fail compute service delete if resource provider delete fails https://review.opendev.org/678100 | |
| 15:49:58 | mriedem | ^ is wip until i get some reviews on what's proposed there which is based on what was discussed in the ML for not orphaning resource providers when deleting a compute service that still has allocations against it | |
| 15:50:15 | mriedem | we continue to get bugs in triage for this kind of thign | |
| 15:50:28 | mriedem | e.g. https://bugs.launchpad.net/nova/+bug/1849701 | |
| 15:50:29 | openstack | Launchpad bug 1849701 in OpenStack Compute (nova) "Resource_provider entry related to a deleted compute node, unable to migrate vms to the node" [Undecided,New] | |
| 16:06:45 | openstackgerrit | John Garbutt proposed openstack/nova-specs master: Add Unified Limits Spec https://review.opendev.org/602201 | |
| 16:09:33 | mriedem | looks like docs job is busted | |
| 16:10:23 | cdent | dansmith, mriedem : when nova-compute comes up and tries to initiate talking to the conductor it uses an "early_timeout" and then after 10 attempts fails over to the global timeout. If those 10 attempts pass, is it possible the nova-compute will sit doing nothing on that long timeout for a long time before talking to the conductor, or should it loop quickly if there is an actual connection failed (rather than timing out | |
| 16:10:23 | cdent | waiting for response)? I'm reading some bugs where it seems like the sit doing nothing is happening, but it seems illogical. | |
| 16:10:24 | mriedem | https://zuul.opendev.org/t/openstack/build/10344924bca44849b06b8fae29b5494f/log/job-output.txt#652 | |
| 16:11:22 | mriedem | cdent: not sure off the top of my head, probably a better question for dansmith | |
| 16:13:25 | cdent | in placement we addressed that docs issue by having a different set of requirements for docs | |
| 16:13:36 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Enable evacuation with qos ports https://review.opendev.org/688688 | |
| 16:14:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Allow evacuating server with port resource request https://review.opendev.org/688387 | |
| 16:14:08 | dansmith | cdent: not sure what you mean by early timeout | |
| 16:14:18 | cdent | dansmith: one sec, will get code link | |
| 16:14:20 | dansmith | cdent: it should be calling ping for the conductor pretty regularly and wait forever | |
| 16:14:23 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Enable evacuation with qos ports https://review.opendev.org/688688 | |
| 16:15:30 | cdent | dansmith: https://github.com/openstack/nova/blob/master/nova/conductor/api.py#L41 | |
| 16:16:05 | dansmith | oh right | |
| 16:16:11 | cdent | after early attempts expires, if the global rpc timeout is huge (because people do that...) it appears like it can be a very long time before "ready" happens | |
| 16:16:43 | cdent | but I would have thought that if the conductor is literally not there, you would know sooner | |
| 16:16:50 | dansmith | cdent: people who set the global timeout large are begging for problems, so I don't think we should do something different here | |
| 16:16:51 | dansmith | however, | |
| 16:17:25 | mriedem | vmware sets the timeout large because they are pulling 1K+ instances from conductor on startup yeah? | |
| 16:17:26 | cdent | but could it be that if the message bus is up but the conductor is not, the ping will hang out in rabbit... waiting | |
| 16:17:28 | dansmith | the reason for that was because if you have a power outage and start a thousand computes, they need to chill if they can't get a response right away.. just because they have a connection doesn't mean they're not overwhelming the conductor(s), which is why we have the backoff | |
| 16:17:44 | dansmith | cdent: sure | |
| 16:17:47 | cdent | mriedem: probably something like that, yeah | |
| 16:18:13 | cdent | Okay, cool, this confirms what I thought was happening, but hadn't been able to trace it all the way down | |
| 16:18:36 | cdent | mriedem: there also appears to be an ordering problem in service restarts | |
| 16:19:01 | cdent | thanks to you both | |
| 16:19:10 | cdent | I'll go write down more stuff that people will ignore... | |
| 16:19:17 | cdent | (not you, I mean internal people) | |
| 16:19:32 | mriedem | *those* people | |
| 16:19:35 | openstackgerrit | John Garbutt proposed openstack/nova master: WIP: Integrating with unified limits https://review.opendev.org/615180 | |
| 16:19:51 | cdent | them | |
| 16:24:10 | mriedem | blarg, yeah we have doc/requirements.txt but https://review.opendev.org/#/c/689152/ threw requirements.txt back into the install for the docs job | |
| 16:24:11 | mriedem | fixing | |
| 16:26:36 | openstackgerrit | Dan Smith proposed openstack/nova master: Log some stats for image pre-cache https://review.opendev.org/688173 | |
| 16:26:37 | openstackgerrit | Dan Smith proposed openstack/nova master: Add Aggregate image caching progress notifications https://review.opendev.org/691390 | |
| 16:29:03 | mriedem | https://bugs.launchpad.net/nova/+bug/1849870 | |
| 16:29:03 | openstack | Launchpad bug 1849870 in OpenStack Compute (nova) "docs build failing on psycopg2 dependency" [Undecided,New] | |
| 16:39:42 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Stop building docs with (test-)requirements.txt https://review.opendev.org/691391 | |
| 16:39:43 | mriedem | ^ will hopefully unblock the gate for nova ^ | |
| 16:41:48 | dansmith | mriedem: should we slam that in? | |
| 16:42:11 | mriedem | let's see if it passes the docs job first, it works for me locally | |
| 16:42:17 | mriedem | guess it could take awhile to get a node... | |
| 16:42:22 | dansmith | okay | |
| 16:42:35 | mriedem | oh it got a node http://zuul.openstack.org/stream/63d6bbd171134e00877ae6ce61856a84?logfile=console.log | |
| 16:42:42 | mriedem | build baby build! | |
| 16:47:39 | mriedem | dansmith: looks like it passed so you can commence to slamming | |
| 16:48:04 | dansmith | mriedem: you going to fix the typo? | |
| 16:48:21 | mriedem | sure, sec | |
| 16:48:22 | dansmith | or just clarifying for review? | |
| 16:48:25 | dansmith | ack | |
| 16:49:03 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Stop building docs with (test-)requirements.txt https://review.opendev.org/691391 | |
| 16:50:32 | openstackgerrit | Dan Smith proposed openstack/nova master: Add Aggregate image caching progress notifications https://review.opendev.org/691390 | |
| 17:00:44 | mriedem | melwitt: were you going to backport https://review.opendev.org/#/c/688206/ and the one below it? | |
| 17:00:55 | mriedem | i'm planning a train release next week so probably good to get that in it | |
| 17:01:14 | melwitt | mriedem: yes. I wasn't sure if I should wait for the top patch to merge first before proposing | |
| 17:01:28 | mriedem | i don't think you need to wait | |
| 17:01:33 | melwitt | ok | |
| 17:09:25 | dansmith | mriedem: what's the trick for generating the notification samples? | |
| 17:32:48 | mriedem | you have to create them manually | |
| 17:33:02 | mriedem | the json file i mean | |
| 17:33:18 | dansmith | WAT | |
| 17:33:50 | mriedem | all sales are final | |
| 17:35:00 | dansmith | if I add this is there something that will check my work? | |
| 17:35:26 | mriedem | the functional test should be comparing against the sample | |
| 17:35:39 | dansmith | oh if I write that test okay | |
| 17:36:04 | dansmith | I should rebase mine on yours I guess | |
| 17:36:05 | mriedem | yeah so instead of this https://review.opendev.org/#/c/691390/2/nova/tests/functional/compute/test_cache_image.py | |
| 17:36:08 | mriedem | right | |
| 17:36:17 | dansmith | not instead, in addition right? | |
| 17:36:26 | mriedem | i don't see why we'd need both | |
| 17:37:01 | dansmith | because it already has the error case and other-agg case | |
| 17:37:04 | mriedem | this is where the test in mine validates the sample against the actual notification generated by the test https://review.opendev.org/#/c/691129/1/nova/tests/functional/notification_sample_tests/test_aggregate.py@199 | |
| 17:37:28 | mriedem | oh yeah ok that makes sense | |
| 17:39:22 | openstackgerrit | Merged openstack/nova stable/train: libvirt: Ignore volume exceptions during post_live_migration https://review.opendev.org/691281 | |
| 17:40:25 | openstackgerrit | melanie witt proposed openstack/nova stable/train: Add regression test for bug 1824435 https://review.opendev.org/691402 | |