| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-19 | |||
| 01:00:48 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellMigrationTask https://review.openstack.org/631581 | |
| 01:00:48 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add TargetDBSetupTask https://review.openstack.org/627892 | |
| 01:00:49 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add can_connect_volume() compute driver method https://review.openstack.org/621313 | |
| 01:00:49 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Execute TargetDBSetupTask https://review.openstack.org/633853 | |
| 01:00:50 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtDestTask https://review.openstack.org/627890 | |
| 01:00:50 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_dest compute method https://review.openstack.org/633293 | |
| 01:00:51 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add nova.compute.utils.delete_image https://review.openstack.org/637605 | |
| 01:00:51 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add prep_snapshot_based_resize_at_source compute method https://review.openstack.org/634832 | |
| 01:00:52 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add PrepResizeAtSourceTask https://review.openstack.org/627891 | |
| 01:01:03 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize conductor RPC method https://review.openstack.org/638047 | |
| 01:01:03 | openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add RevertResizeTask https://review.openstack.org/638046 | |
| 01:01:04 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize while deleting a server https://review.openstack.org/638268 | |
| 01:01:04 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert cross-cell resize from the API https://review.openstack.org/638048 | |
| 01:01:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.openstack.org/614353 | |
| 01:01:05 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add archive_deleted_rows wrinkle to cross-cell functional test https://review.openstack.org/651650 | |
| 01:01:06 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add cross-cell resize policy rule and enable in API https://review.openstack.org/638269 | |
| 01:55:15 | openstackgerrit | melanie witt proposed openstack/nova master: Fix SynchronousThreadPoolExecutorFixture mock spec https://review.openstack.org/650171 | |
| 01:55:16 | openstackgerrit | melanie witt proposed openstack/nova master: Revert "Fix target_cell usage for scatter_gather_cells" https://review.openstack.org/653894 | |
| 01:55:16 | openstackgerrit | melanie witt proposed openstack/nova master: Use futurist.GreenThreadPoolExecutor in scatter_gather_cells https://review.openstack.org/650172 | |
| 04:02:15 | eandersson | efried, found something odd with the ironic code | |
| 04:02:25 | eandersson | http://logs.openstack.org/39/653839/2/check/ironic-tempest-ipa-wholedisk-bios-agent_ipmitool-tinyipa/63a832a/controller/logs/screen-n-cpu.txt.gz#_Apr_19_03_21_25_243777 | |
| 04:02:30 | eandersson | https://review.openstack.org/#/c/653839/2/nova/virt/ironic/client_wrapper.py | |
| 04:02:42 | eandersson | If you provide a region_name, the get_endpoint does not seem to work | |
| 04:03:40 | eandersson | It looks to be related to min_version, max_version. If I remove those options it works as intended. | |
| 06:56:35 | openstackgerrit | ya.wang proposed openstack/nova-specs master: Expose auto converge and post copy https://review.openstack.org/651681 | |
| 07:38:15 | lpetrut | Hi. Is nova already using nested resource providers for NUMA topologies? looking at the libvirt driver 'update_provider_tree' method it seems not but I may be wrong | |
| 08:00:34 | kashyap | cfriesen: I think you didn't read the Secure Boot spec in full. I did mention in the Work Items that we will re-use existing flavor/image properties. | |
| 08:57:50 | kashyap | cfriesen: Anyway, will respond to your questions on the change :-) | |
| 09:38:57 | openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Specifying az when restore shelved server https://review.openstack.org/624689 | |
| 09:47:52 | openstackgerrit | ya.wang proposed openstack/nova-specs master: Expose auto converge and post copy https://review.openstack.org/651681 | |
| 09:52:52 | openstackgerrit | Boxiang Zhu proposed openstack/nova master: Add host and hypervisor_hostname flag to create server https://review.openstack.org/645520 | |
| 11:49:31 | yan0s | hello I'm having a problem with VNC in the web gui | |
| 11:49:50 | yan0s | I can see it using the admin user | |
| 11:50:03 | yan0s | but not with "Member" role users (also using SAML integrated login) | |
| 11:50:13 | yan0s | error in nova-novncproxy.log is : code 400, message Bad request syntax ('\x88\x8fI¿cØJW7¹;Ø\x06¬iÜ\x0f·:Ú\x07') | |
| 13:05:30 | mriedem | seems like something exploded in infra over night and now everything has to be rechecked | |
| 13:06:19 | yan0s | Actually, novnc breaks only when I launch the VM with a cloud-config file | |
| 13:25:12 | fried_rice | eandersson: I can't say I'm particularly surprised tbh. | |
| 13:25:59 | fried_rice | do you have a way of seeing whether it broke recently? | |
| 13:29:36 | fried_rice | mriedem: TestRPC.test_create_transport in py36 - is that the thing you were seeing yesterday? | |
| 13:29:45 | mriedem | yes | |
| 13:29:47 | mriedem | see ML | |
| 13:33:53 | fried_rice | thanks mriedem. Recheckable or 100%? | |
| 13:42:48 | mriedem | i've been rechecking | |
| 13:43:05 | mriedem | with 12 hour turnaround on ci times plus there is a gerrit maintenance scheduled today... | |
| 13:43:11 | mriedem | looks like i should just quite early | |
| 13:47:09 | fried_rice | mriedem: When I run this locally, I get 'oslo_config.cfg.NoSuchGroupError: no such group [oslo_messaging_notifications]' | |
| 13:47:58 | fried_rice | Trying to find where that group is being registered... | |
| 13:48:31 | mriedem | via oslo.messaging opts | |
| 13:48:59 | mriedem | i think in nova.rpc | |
| 13:49:09 | mnaser | mriedem: also, I had to turn down ovh-bhs1 (150 VMs) because it had networking issues | |
| 13:49:42 | mnaser | and now I'm seeing a lot of RETRY_LIMIT failing jobs, which means that another provider is having issues (I suspect the other ovh region from what I was seeing) | |
| 13:49:43 | mriedem | ah this https://github.com/openstack/nova/blob/master/nova/config.py#L50 | |
| 13:50:02 | mriedem | mnaser: yeah i saw a lot of infra failures like that | |
| 13:50:13 | mnaser | so we're pretty short on infra :\ | |
| 13:50:30 | mnaser | but 100% relate on the "lets fix our gates that are on fire instead of making code pretty" comment | |
| 13:50:38 | mriedem | mnaser: has there been something else going on all week because it's taken half a day to get a result on a change at times | |
| 13:51:25 | mnaser | mriedem: afaik ovh has been having issues where a whole job will run and then fail to collect logs, restarting that job | |
| 13:51:35 | mnaser | so in our case a 2-3h deploy job would run 3 times in a row and still fail | |
| 13:51:56 | mriedem | ah | |
| 13:52:00 | mnaser | and even then it'd re-queue after the fail so takes ages to get a node assigned again | |
| 13:52:05 | mriedem | yeah i saw the POST_FAILURES which is generally failing to collect and publish logs | |
| 13:52:14 | mnaser | because there's so many other things that are trying to also retry | |
| 13:53:03 | mriedem | fried_rice: or it might be this that registers those options https://github.com/openstack/nova/blob/master/nova/config.py#L61 | |
| 13:53:23 | mnaser | and today you're going to have an automated patch change all the references in your code to point to opendev and so | |
| 13:53:30 | mnaser | enjoy that it in advance and hopefully that doesn't break your world too | |
| 13:58:18 | mriedem | which core can i pay off to look at these 2 simple changes? https://review.openstack.org/#/q/topic:bug/1823781+(status:open+OR+status:merged) | |
| 14:00:02 | fried_rice | I'll look. But your credit score is shit with me. "The nickel is in my backpack," suuuure. | |
| 14:00:42 | fried_rice | You were just seeing if we were paying attention. | |
| 14:00:42 | fried_rice | Berlin." | |
| 14:00:42 | fried_rice | PTG discussions are fresh. Just a couple months until the Summit and Forum in | |
| 14:00:42 | fried_rice | tonyb[m]: "Now would be a good time to start brainstorming Forum topics while some of the | |
| 14:01:25 | mriedem | fried_rice: it's still in there, i just can't ever remember to give it to you in person | |
| 14:01:31 | mriedem | my offer to mail it still stands | |
| 14:02:16 | fried_rice | Then it'll be, "the nickel's in the mail." I've heard that one before too. | |
| 14:04:19 | fried_rice | I get that same NoSuchGroupError in stein and rocky. So I'm guessing there's a bug in the test setup where the test env doesn't run through the code path that registers the options if I'm just running those two tests. | |
| 14:05:55 | mriedem | https://photos.app.goo.gl/giiB1e8RD8WztkaQ8 | |
| 14:05:57 | mriedem | i've set a reminder | |
| 14:06:23 | mriedem | fried_rice: yeah i get the same: $ tox -r -e py36 -- nova.tests.unit.test_rpc --until-failure | |
| 14:07:38 | fried_rice | so that's a red herring for our gate failure :( | |
| 14:12:12 | fried_rice | If you actually pay me off, I won't be able to keep giving you crap about it. We need a different plan. | |
| 14:14:46 | mriedem | what i'd like to know is what is calling oslo.config when the death spiral starts | |
| 14:23:22 | mriedem | well i fixed that latent traceback at least :) | |
| 14:23:25 | fried_rice | Running all of py36 locally I got these: | |
| 14:23:26 | fried_rice | {7} nova.tests.unit.objects.test_instance_group.TestInstanceGroupObject.test_get_by_hint [] ... inprogress | |
| 14:23:26 | fried_rice | {2} nova.tests.unit.pci.test_utils.PciDeviceMatchTestCase.test_spec_extra_key [] ... inprogress | |
| 14:24:53 | fried_rice | oh, by the way, that ^ was on rocky | |
| 14:25:54 | fried_rice | what have we backported lately? | |
| 14:26:57 | mriedem | shrug, lots of stuff | |
| 14:27:07 | mriedem | what is getting an option with a namespace? https://github.com/openstack/oslo.config/blob/6.8.1/oslo_config/cfg.py#L2614 | |
| 14:27:12 | mriedem | wtf is a config option namespace? | |
| 14:27:15 | mriedem | stephenfin: ^? | |
| 14:27:31 | mriedem | http://logs.openstack.org/45/649345/7/check/openstack-tox-py36/ba15c17/job-output.txt.gz#_2019-04-18_18_10_53_423952 | |
| 14:27:37 | mriedem | that's where the stack overflow starts | |
| 14:28:24 | fried_rice | when rebuilding my venv?? | |
| 14:28:24 | fried_rice | Error: pg_config executable not found. | |
| 14:28:24 | fried_rice | and why am I suddenly getting | |
| 14:28:46 | mriedem | https://github.com/openstack/oslo.config/blob/6.8.1/oslo_config/cfg.py#L2183 | |
| 14:29:33 | mriedem | might be due to newer psycopg2? | |
| 14:29:42 | mriedem | are you on 18.04? | |