| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-12-13 | |||
| 16:09:22 | dansmith | yar | |
| 16:09:25 | mriedem | i think before rocky, devstack didn't use upgrade_levels=auto | |
| 16:09:34 | dansmith | ack | |
| 16:10:08 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Make compute rpcapi version calculation check all cells https://review.openstack.org/624982 | |
| 16:25:22 | mriedem | dansmith: hmm, thinking about that... | |
| 16:25:34 | mriedem | people are setting api db in cell configs, | |
| 16:25:37 | mriedem | so they can do the up-call stuff | |
| 16:25:54 | mriedem | so when nova-compute starts up in cell1, it could be determining it's min version from computes in cell50 | |
| 16:25:55 | dansmith | as long as it's the same on the compute and corresponding conductor it'll befine | |
| 16:25:56 | mriedem | in cerns case | |
| 16:26:06 | dansmith | oh | |
| 16:26:21 | dansmith | I see what you mean, but that probably isn't terrible | |
| 16:26:40 | dansmith | if you're giving the cell visibility to the whole deployment, then .. you get the whole deployment's min version | |
| 16:26:52 | dansmith | this is exactly why I *don't* want to make this a common practice, fwiw, | |
| 16:27:05 | dansmith | because now you're talking about teaching the cells about their place in the world and that gets icky | |
| 16:28:04 | mriedem | i just don't know if it could throw anyone off, i.e. upgrade reno or anything, | |
| 16:28:11 | mriedem | but if you're doing upgrade_levels=auto, | |
| 16:28:19 | mriedem | you're opting into the system figuring this out | |
| 16:28:25 | mriedem | and can pin manually if you don't trust the system | |
| 16:28:42 | mriedem | let's pretend i never mentioned this | |
| 16:28:59 | dansmith | yep | |
| 16:35:22 | openstack | Launchpad bug 1808063 in OpenStack-Gate "glanceclient.exc.HTTPBadGateway: 502 Proxy Error during server snapshot" [Undecided,Confirmed] | |
| 16:35:22 | cdent | mriedem: I've added some links to https://bugs.launchpad.net/openstack-gate/+bug/1808063 for the recent failure that you rechecked. g-api is having a stackoverflow | |
| 16:38:45 | mriedem | oh nice | |
| 16:38:46 | mriedem | http://logs.openstack.org/41/617941/31/gate/grenade-py3/7909f30/logs/screen-g-api.txt.gz#_Dec_13_14_43_21_414511 | |
| 16:38:50 | mriedem | Dec 13 14:43:21.414511 ubuntu-xenial-rax-ord-0001176665 devstack@g-api.service[6990]: File "/usr/local/lib/python3.5/dist-packages/oslo_messaging/_drivers/impl_rabbit.py", line 926 in _heartbeat_thread_job | |
| 16:39:03 | mriedem | dansmith: look familiar? ^ | |
| 16:39:10 | cdent | and a second comment | |
| 16:39:25 | mriedem | File "/opt/stack/new/glance/glance/quota/__init__.py", line 168 in __getattr__ looks like infinite loop? | |
| 16:39:33 | cdent | eyes | |
| 16:39:35 | cdent | yes | |
| 16:40:03 | mriedem | https://github.com/openstack/glance/commit/cb45edf5c81f7d09c9ef0b88d40d56b4750beb10#diff-5afd3797ee6a482f2c0d4862140f7014 was the last change in there but awhile ago | |
| 16:40:42 | mriedem | https://github.com/openstack/glance/blob/master/glance/quota/__init__.py#L168 | |
| 16:44:54 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Only warn about not having computes nodes once in rpcapi https://review.openstack.org/625051 | |
| 17:02:12 | melwitt | mriedem: +1 to flushing queens and pike backports and releasing before the holiday break. I assumed not rocky bc we did that semi recently but were you thinking rocky too? I'm sure a lot has landed since last time | |
| 17:02:37 | melwitt | I will help flush | |
| 17:11:22 | mriedem | melwitt: yes otherwise we could release things in pike/queens that are not yet released in rocky | |
| 17:11:31 | mriedem | generally need to snowball these from newest to oldest branch | |
| 17:11:44 | melwitt | good point, I should have thought of that. thanks | |
| 17:12:07 | openstackgerrit | Merged openstack/nova stable/rocky: Handle tags in _bury_in_cell0 https://review.openstack.org/624813 | |
| 17:14:31 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/queens: Make compute rpcapi version calculation check all cells https://review.openstack.org/625060 | |
| 17:35:59 | openstackgerrit | melanie witt proposed openstack/nova master: WIP Add configuration of maximum disk devices to attach https://review.openstack.org/616777 | |
| 17:35:59 | openstackgerrit | melanie witt proposed openstack/nova master: Add method to generate device names universally https://review.openstack.org/573066 | |
| 17:35:59 | openstackgerrit | melanie witt proposed openstack/nova master: Propagate exception message from _prep_block_device https://review.openstack.org/624832 | |
| 17:40:47 | colby_ | Build of instance aborted: Nova does not support Cinder API version 2. The catalog is set to use v3 cinder as default, and in the nova config we have this set for cinder: catalog_info=volumev3:cinderv3:internalURL | |
| 17:40:47 | colby_ | Hey Eveyrone. We just upgraded to queeens. Im seeing some weird issues when creating an instance from an already created volume. Im seeing this error: | |
| 17:41:06 | colby_ | we have a volumev3 in the catalog as well that points to the v3 api | |
| 17:55:20 | melwitt | colby_: is it possible you have [cinder]endpoint_template set, which will override catalog_info? | |
| 18:01:55 | cezary_zukowski | Hey, how to simulate/mimic a network disruption occuring between a Nova instance/server and Cinder (volume/storage)? Does detaching a volume from instance manually would be a sensible approach/substitute? | |
| 18:01:57 | mriedem | that would be my guess, endpoint_template is set | |
| 18:02:20 | cezary_zukowski | Command for detaching: nova volume-detach <instance_id> <volume_id> | |
| 18:02:48 | cezary_zukowski | mriedem: What do you mean? | |
| 18:03:38 | colby_ | melwitt: no its not set in the config anywhere | |
| 18:03:46 | mriedem | cezary_zukowski: was answering a different question | |
| 18:03:56 | cezary_zukowski | mriedem: ah, right | |
| 18:04:29 | mriedem | colby_: do you have this? https://github.com/openstack/nova/commit/c7d87f6691bbc39daca918729b769cc2d2000abd | |
| 18:04:45 | mriedem | that's in 17.0.6 | |
| 18:04:50 | mriedem | if you're queens ga you wouldn't have that | |
| 18:05:13 | mriedem | not sure if that's your problem though | |
| 18:17:42 | melwitt | colby_: aside from all of that, I would just take a look at your keystone catalog again and double check there's no /v2.0 for nova (should be /v2.1) and no /v2 for cinder | |
| 18:18:33 | melwitt | because the only other thing I see the code doing to determine the cinder version is parsing the version from the endpoint found in the catalog | |
| 18:18:45 | colby_ | oh it has the wrong config on the actual compute nodes. Thanks that helped me narrow it down. Ill switch that config there. Apparently it just got switched on the controller services | |
| 18:20:14 | melwitt | ok, cool | |
| 18:45:35 | mpiwowarczy | hello | |
| 18:46:36 | mpiwowarczy | can I have a question to nova deprecated personality parameter? | |
| 18:47:14 | mpiwowarczy | seems it will be removed in microversion 2.57 | |
| 18:47:42 | mpiwowarczy | but can not found when 2.57 will be used? | |
| 18:47:55 | mpiwowarczy | for Rocky release it was microversion 2.27 | |
| 18:48:06 | mpiwowarczy | which version will be used in Stein? | |
| 18:49:01 | mpiwowarczy | or is there any fixed date when 2.57 version will be dropped? | |
| 18:54:22 | artom | mpiwowarczy, microversions are never dropped, we still support 2.1 if the client asks for it | |
| 18:54:34 | artom | mpiwowarczy, in terms of releases, have a look at https://docs.openstack.org/nova/latest/reference/api-microversion-history.html | |
| 18:56:49 | mpiwowarczy | artom: thanks for the answer | |
| 18:56:56 | mpiwowarczy | so when personality files (files injection) feature will be dropped? | |
| 18:58:11 | mpiwowarczy | will be this supported in Stein release? | |
| 19:01:35 | artom | mpiwowarczy, it'll never be dropped per se | |
| 19:02:05 | artom | mpiwowarczy, you can keep using it in Stein if the client asks for a microversion < 2.57 | |
| 19:03:49 | mpiwowarczy | ok, thanks for your help | |
| 19:07:37 | melwitt | mpiwowarczy: it's dropped in a sense that if/when you find yourself needing to request microversion >= 2.57 when you call the REST API to get a different feature/addition, you will not be able to use personality files at the same time/in the same call | |
| 19:08:40 | mpiwowarczy | melwitt: I see | |
| 19:10:15 | mpiwowarczy | do you intend to maintain all versions, starting from 2.1, in all future OpenStack releases? | |
| 19:11:22 | mpiwowarczy | day by day it is more difficult to maintain :) | |
| 19:18:33 | mriedem | what is more difficult to maintain? writing apps against the API? or maintaining the various microversions in the API? | |
| 19:18:54 | mriedem | the client opts into the behavior, so if you are ok with v2.1 then you don't need to ever change that | |
| 19:20:01 | mpiwowarczy | oh yeah, you're right | |
| 19:20:49 | mpiwowarczy | just one more question - latest novaclient 11.1.0, which microversion it uses? | |
| 19:20:59 | mriedem | the alternative is we don't use microversions, change the api from release to release, and all client software everywhere in the world has to deal with that based on the cloud they are talking to | |
| 19:21:00 | mpiwowarczy | where I can check that? | |
| 19:21:27 | mriedem | the client-side support is defined here https://github.com/openstack/python-novaclient/blob/master/novaclient/__init__.py#L28 | |
| 19:21:35 | mriedem | novaclient python APIs by default use 2.1 | |
| 19:21:51 | mriedem | the CLI by default negotiates the highest supported microversion between the client and the server | |
| 19:22:24 | mriedem | so if you're using novaclient as a python sdk, then it's opt-in, | |
| 19:22:37 | mriedem | if you're using the CLI, it's negotiated unless you tell it otherwise | |
| 19:23:09 | mpiwowarczy | it is about SDK | |
| 19:23:14 | mriedem | then it's opt-in | |
| 19:23:27 | mpiwowarczy | we have defined version python-novaclient==11.1.0 and we use file injection | |
| 19:23:32 | mriedem | api methods in novaclient are wrapped / bound based on what versions they support | |
| 19:23:49 | mriedem | as i said, unless you opt into a newer microversion, it will use 2.1 | |
| 19:23:51 | mriedem | for the sdk | |
| 19:23:55 | openstackgerrit | Merged openstack/nova stable/rocky: Harden placement init under wsgi https://review.openstack.org/617297 | |