| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-07-11 | |||
| 21:20:20 | mriedem | cern has bm and they are on queens now, | |
| 21:20:39 | mriedem | tssurya: do you know if the cern baremetal deployment has gone through the custom resource class transition for ironic flavors? | |
| 21:21:18 | tssurya | mriedem: yes | |
| 21:21:31 | jroll | \o/ | |
| 21:22:20 | jroll | mriedem: I can get back to 565841 next week maybe, or someone can take it over if they want | |
| 21:22:42 | tssurya | meaning we use resource classes for ironic flavors (reading the convo to find out what transition) | |
| 21:23:21 | jroll | tssurya: there's an online data migration for existing instances | |
| 21:23:23 | edmondsw | mriedem I think your changes in https://review.openstack.org/#/c/528835/ are related to a bug I just opened... take a look when you get a chance? https://bugs.launchpad.net/nova/+bug/1781286 | |
| 21:23:25 | openstack | Launchpad bug 1781286 in OpenStack Compute (nova) "CantStartEngineError in cell conductor during rebuild" [Undecided,New] | |
| 21:23:40 | TheJulia | jroll: mriedem: I'm sure some folks have in our org... That being said unless it is a stated policy I've noticed people upstream tend to try and push the envelope... and downstream seems to pay for it... I guess I'm just rambling that long, but not too long is preferable to short windows | |
| 21:24:18 | jroll | TheJulia: dropping it now would be two cycles | |
| 21:25:07 | mriedem | TheJulia: downstream consumers would have upstream maintain everything forever if possible | |
| 21:25:09 | TheJulia | to support skip-leveling, two would be necessary, right now most people seem to be focused on fast forwarding since that is essentially what we end up testing | |
| 21:25:22 | TheJulia | mriedem: :( | |
| 21:25:30 | TheJulia | mriedem: I do not disagree | |
| 21:25:36 | mriedem | one release is necessary for FFU | |
| 21:25:39 | mriedem | there is no skip level | |
| 21:25:58 | jroll | oh there is skip level :P | |
| 21:26:05 | mriedem | FFU means rolling through each release, not going directly from ocata to queens | |
| 21:26:13 | TheJulia | jroll: heh | |
| 21:26:14 | mriedem | sure, there is, but it's totally undefined | |
| 21:26:19 | mriedem | and therefore impossible to support | |
| 21:26:28 | jroll | right | |
| 21:26:52 | jroll | and having to deal with online data migrations gone missing in skip level is already an issue | |
| 21:27:02 | mriedem | given we're close to the end of rocky, i don't know if we just punt until early stein for the big drop or do it now | |
| 21:27:24 | TheJulia | Anyway folks, I'm going to go like... enjoy... the big blue room. | |
| 21:27:33 | jroll | at any rate, there's a patch up that needs rebasing, I'm fine either way | |
| 21:27:34 | mriedem | we might as well just wait | |
| 21:27:35 | TheJulia | +1 to punting | |
| 21:28:01 | jroll | mmmkay | |
| 21:28:53 | TheJulia | lol | |
| 21:32:31 | openstackgerrit | Merged openstack/nova master: Add queued for delete to instance_mappings table. https://review.openstack.org/566788 | |
| 21:32:47 | openstackgerrit | Merged openstack/nova master: Fix TypeError in prep_resize allocation cleanup https://review.openstack.org/581548 | |
| 21:32:52 | mriedem | jroll: ok added to https://etherpad.openstack.org/p/nova-ptg-stein | |
| 21:33:00 | openstackgerrit | Merged openstack/nova master: Add functional regressions tests for server_group_members OverQuota https://review.openstack.org/580755 | |
| 21:33:07 | openstackgerrit | Merged openstack/nova master: Fix server_group_members quota check https://review.openstack.org/580684 | |
| 21:33:56 | mriedem | i have no idea what the big blue room is | |
| 21:34:25 | mriedem | oh i see, the cool way to say go outside | |
| 21:37:29 | mriedem | edmondsw: "The nova_cell1.conf does have [database]connection set:" - that's not the db we're failing on | |
| 21:37:43 | mriedem | edmondsw: build requests are in the api db, which devstack won't configure for n-cond-cell1 | |
| 21:37:51 | mriedem | since the cell shouldn't be able to reach the api db | |
| 21:38:07 | mriedem | edmondsw: if you have a recreate, | |
| 21:38:24 | mriedem | can you add a debug line to dump filter_properties? | |
| 21:38:25 | edmondsw | yeah, I thought it might be the api db from a comment in your other change | |
| 21:38:26 | mriedem | because | |
| 21:38:26 | mriedem | num_attempts = filter_properties.get( | |
| 21:38:26 | mriedem | 'retry', {}).get('num_attempts', 1) | |
| 21:38:36 | edmondsw | otp right now, will check back in a min | |
| 21:38:43 | mriedem | is it a rolm phone? | |
| 21:51:53 | edmondsw | mriedem rolm? | |
| 21:52:14 | edmondsw | I think I can get the data you asked for | |
| 21:52:15 | mriedem | http://www.wikicu.com/ROLM_Phone | |
| 21:52:24 | mriedem | edmondsw: are you by chance forcing the host/node during server create | |
| 21:52:33 | mriedem | using --availability-zone nova:host:node? | |
| 21:52:36 | mriedem | something like that? | |
| 21:52:51 | edmondsw | not that I know of, but I'm not sure | |
| 21:53:02 | edmondsw | this is in the PowerVM CI environment | |
| 21:53:05 | mriedem | well, are you issuing a nova boot? | |
| 21:53:10 | mriedem | oh... | |
| 21:53:12 | edmondsw | so it's all automatated (hit this while redeploying it) | |
| 21:53:25 | edmondsw | so I'm not sure what it's doing without digging into the automation | |
| 21:53:37 | edmondsw | this is the undercloud | |
| 21:54:01 | mriedem | ok i guess check to see if it's forcing a host/node | |
| 21:54:07 | edmondsw | will do | |
| 21:54:21 | mriedem | or if CONF.scheduler.max_attempts = 1 | |
| 21:55:47 | edmondsw | max_attempts is the default (3) | |
| 21:55:56 | mriedem | ok | |
| 21:59:12 | edmondsw | mriedem I think these instances are being created by nodepool, if that tells you anything | |
| 22:00:31 | mriedem | i'm going to see if i can recreate with a functional test quick | |
| 22:00:35 | edmondsw | I doubt we're specifying a specific host/node but I'm looking | |
| 22:00:36 | mriedem | but no nodepool doesn't tell me anything | |
| 22:00:50 | openstackgerrit | Takashi NATSUME proposed openstack/nova-specs master: Create specs directory for Stein https://review.openstack.org/573602 | |
| 22:01:09 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (5) https://review.openstack.org/570842 | |
| 22:01:10 | edmondsw | and whether we do or not, I don't know whether nodepool would | |
| 22:01:36 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (6) https://review.openstack.org/571330 | |
| 22:02:06 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (7) https://review.openstack.org/571992 | |
| 22:02:38 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (8) https://review.openstack.org/571993 | |
| 22:06:44 | edmondsw | mriedem I think this is an example request body: body: {"server": {"name": "PowerVM_CI-PowerVM_DevStacked-440", "imageRef": "0dd943fd-ae60-4cdf-85fb-709d00f87db1", "flavorRef": "55b0569d-4f51-47fd-a326-8b333fedcce3", "max_count": 1, "min_count": 1, "networks": [{"uuid": "a4a8157c-da2f-462f-aa73-5c97fc7e17f1"}], "metadata": {"nodepool": "{\"provider_name\": \"PowerVM_DevStacked\", \"node_id\": 440, \"image_name\": \" | |
| 22:06:44 | edmondsw | PowerVM_Ubuntu_Base\"}", "groups": "[\"PowerVM_DevStacked\", \"PowerVM_Ubuntu_Base\"]"}}} {{(pid=31995) _process_stack /opt/stack/nova/nova/api/openstack/wsgi.py:604}} | |
| 22:07:32 | edmondsw | no availability zone specified | |
| 22:11:28 | mriedem | i don't know what \"node_id\": 440 is | |
| 22:13:54 | edmondsw | mriedem I think that's just something nodepool wants to remember, not important to nova | |
| 22:14:15 | edmondsw | mriedem the Exception block you pasted in comment 1 is not entered | |
| 22:14:26 | edmondsw | the error isn't until after that | |
| 22:18:40 | mriedem | ok log local_filter_props before the other instance of calling _destroy_build_request | |
| 22:21:48 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (3) https://review.openstack.org/574104 | |
| 22:22:05 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (4) https://review.openstack.org/574106 | |
| 22:22:20 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (5) https://review.openstack.org/574110 | |
| 22:23:31 | mriedem | edmondsw: what is the devstack env like? a single host but with multiple nova-compute services running the powervm driver? | |
| 22:23:37 | mriedem | so a 1:M host:node setup? | |
| 22:23:53 | edmondsw | local_filter_props isn't set until a few lines after the error | |
| 22:24:24 | edmondsw | the environment is a controller that runs most things (api, single conductor, etc.) and then multiple compute nodes | |
| 22:24:58 | edmondsw | the multiple compute nodes are all on actual physical nodes | |
| 22:25:08 | edmondsw | since this is the undercloud | |
| 22:25:34 | mriedem | oh shite i see, | |
| 22:25:34 | mriedem | get_host_availability_zone | |
| 22:25:41 | mriedem | that's an up-call to get aggregates for the host | |
| 22:25:43 | mriedem | to find the az | |
| 22:27:45 | mriedem | so we reschedule, pop another host off the selection list, | |
| 22:28:02 | mriedem | and then call host | |