| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-13 | |||
| 13:00:45 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: Plumbing required in servers ViewBuilder to construct partial results https://review.openstack.org/635146 | |
| 13:00:46 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells Documentation https://review.openstack.org/635147 | |
| 13:13:29 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells Documentation https://review.openstack.org/635147 | |
| 13:15:22 | openstackgerrit | Jim Rollenhagen proposed openstack/nova master: ironic: partition compute services by conductor group https://review.openstack.org/635006 | |
| 13:18:27 | openstackgerrit | Jim Rollenhagen proposed openstack/nova master: Ensure config regexes match the entire string https://review.openstack.org/636627 | |
| 13:22:24 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Change nova-next tempest test regex https://review.openstack.org/636459 | |
| 13:28:12 | Shilpa | stephenfin: hi, under Taskflow, added https://bugs.launchpad.net/taskflow/+bug/1815738, do let me know is this active channel? Unable to trace PTL for this, but https://pypi.org/project/taskflow/ reports latest release in Jan'19 | |
| 13:28:13 | openstack | Launchpad bug 1815738 in taskflow "Taskflow doesn't support to retrieve tasks progress details in the order of workflow task execution" [Undecided,New] | |
| 13:30:09 | mriedem | Shilpa: i think that would be in the #openstack-oslo channel now | |
| 13:31:11 | mriedem | or i guess #openstack-state-management https://wiki.openstack.org/wiki/TaskFlow#Contact_us.21 | |
| 13:31:26 | mriedem | tssurya: did you enjoy my tales from going down cell down street? | |
| 13:31:53 | Shilpa | mriedem: #openstack-state-management here no one responding, will check #openstack-oslo channel | |
| 13:32:14 | mriedem | tssurya: belmoreira: i'm interested - what do you configure [upgrade_levels]/compute to in nova.conf? do you pin to a release or have it set to auto? | |
| 13:32:23 | bauzas | mriedem: FWIW, I uploaded the new rev for reshape | |
| 13:32:36 | bauzas | mriedem: I found a potential blocker, but eventually it's not a problem | |
| 13:32:55 | mriedem | ok cool | |
| 13:33:58 | belmoreira | mriedem: in the past used to pin it to a release. Now we use auto | |
| 13:36:56 | openstackgerrit | Lee Yarwood proposed openstack/nova master: Avoid redundant initialize_connection on source post live migration https://review.openstack.org/551302 | |
| 13:37:37 | mriedem | belmoreira: ok if you have this change from stable/rocky https://review.openstack.org/#/c/624982/ then you might want to be careful, | |
| 13:37:43 | mriedem | i came across something nasty last night in down cell testing | |
| 13:38:02 | mriedem | https://bugs.launchpad.net/nova/+bug/1815697 | |
| 13:38:03 | openstack | Launchpad bug 1815697 in OpenStack Compute (nova) "[upgrade_levels]compute=auto grinds the API response times when a cell is down" [Medium,Confirmed] | |
| 13:38:52 | mriedem | at this point i don't have any great solutions for that bug | |
| 13:40:00 | sean-k-mooney | mriedem: as a workaournd you set the retry interval to 1 and retries to 1 and set the exact version | |
| 13:40:29 | sean-k-mooney | that could lead to more api request failing in other cases but it should not be inherintly dangourous right | |
| 13:40:32 | mriedem | sean-k-mooney: no, the max_retries and retry_interval was just to quiet the logs | |
| 13:40:39 | sean-k-mooney | oh ok | |
| 13:40:46 | mriedem | the workaround for hanging the api on startpu was setting [upgrade_levels]/compute=rocky | |
| 13:42:11 | tssurya | mriedem: I did go through your tales :D I absolutely appreciate the testing :) I am adding more test cases for the parts you felt are lacking | |
| 13:42:51 | belmoreira | mriedem: we don't have that commit, yet. Thanks for the heads up. | |
| 13:42:53 | tssurya | and I am not so sure I get your worry here: https://review.openstack.org/#/c/591657/39/nova/api/openstack/compute/servers.py@135 | |
| 13:43:38 | Shilpa | mriedem: no one replies here too #openstack-oslo, will wait for more time | |
| 13:45:41 | mriedem | Shilpa: ok well nova doesn't use taskflow. harlowja was the primary maintainer but i'm not sure if he's around anymore. oslo team would be your best bet i'd think, otherwise maybe #openstack-dev | |
| 13:45:58 | mriedem | efried loves taskflow though | |
| 13:46:49 | Shilpa | mriedem: ok, hope he will help me there | |
| 13:47:37 | mriedem | tssurya: my concern is if tooling is passing filter parameters by default, | |
| 13:47:50 | mriedem | normally in the API we filter out search options that the user isn't allowed to use without it being an error | |
| 13:48:18 | mriedem | so let's say i'm a non-admin and i pass some stuff which normally just gets filtered out in the remove_invalid_options method | |
| 13:48:25 | mriedem | essentially making search_opts = {} | |
| 13:48:41 | tssurya | like "deleted" and "tenant" ? | |
| 13:48:46 | mriedem | if i'm using 2.69, then empty search_opts would mean down_cell_support=True | |
| 13:48:59 | mriedem | tssurya: non-admins can't use the tenant filter | |
| 13:49:10 | tssurya | yea I know | |
| 13:49:12 | mriedem | servers are filtered by default from the project_id in the request context | |
| 13:49:27 | tssurya | which is why I was asking if that's what you mean by invalid options | |
| 13:50:04 | mriedem | so let's say i'm a non-admin and i'm using --deleted, here are some scenarios | |
| 13:50:25 | mriedem | 1. nova list --deleted with v2.1 - the deleted filter would be excluded w/o any errors in the api and i should list my servers | |
| 13:50:51 | mriedem | 2. nova list --deleted with v2.69 and list_records_by_skipping_down_cells=True (default), the down_cell_support flag would be set to False and i wouldn't get any results (if my servers are in a down cell) | |
| 13:51:06 | mriedem | 3. nova list --deleted with v2.69 and list_records_by_skipping_down_cells=False, if my servers are in a down cell i'll get a 500 | |
| 13:51:45 | mriedem | my point was if we calculated the down_cell_support flag *after* calling remove_invalid_options to filter the search_opts, then #3 becomes #2 | |
| 13:52:00 | mriedem | actually no that's not right, | |
| 13:52:14 | mriedem | i'd get partial results | |
| 13:53:14 | openstackgerrit | Lee Yarwood proposed openstack/nova master: Restore connection_info after live migration rollback https://review.openstack.org/551349 | |
| 13:53:16 | mriedem | to summarize, i think the down_cell_support variable should be set based on the search opts after we've filtered that set in remove_invalid_options to increase the chances of someone getting partial results | |
| 13:53:17 | lyarwood | mdbooth: ^ btw, feel free to respin the commit message if you have time. | |
| 13:53:55 | tssurya | mriedem: ok, I get what you mean | |
| 13:54:21 | mriedem | the tricky thing is if you do that, you have to explicitly look for paging and sorting parameters | |
| 13:54:30 | mriedem | because of https://review.openstack.org/#/c/591657/39/nova/api/openstack/compute/servers.py@1235 | |
| 13:54:38 | tssurya | yea | |
| 13:54:55 | mriedem | but i dont think that's so hard, we could just create a constant for ('sort_key', 'sort_dir', 'limit', 'marker') and use that in both places | |
| 13:55:34 | mriedem | and your _is_cell_down_supported method can check, "if any(PAGING_SORTING_PARAMS in reg.GET) | |
| 13:57:01 | tssurya | got it, let me dig more and incorporate that | |
| 13:57:17 | mriedem | err, if set(PAGING_SORTING_PARAMS) & set(list(req.GET.keys())) | |
| 13:57:20 | mriedem | something like that | |
| 13:57:33 | mriedem | does that make sense though? | |
| 13:57:45 | tssurya | it makes sense yea | |
| 13:57:59 | openstackgerrit | Hamdy Khader proposed openstack/os-vif master: Add create_ovs_port field in VIFPortProfileOpenVSwitch profile https://review.openstack.org/636061 | |
| 13:58:01 | tssurya | but I still don't get why the policy things are mixed with ignoring flags | |
| 13:58:06 | mriedem | cool. my goal is to maximize the chance that someone gets partial results if they have servers in a down cell | |
| 13:58:29 | mriedem | which policy things? | |
| 13:58:37 | tssurya | I mean if --deleted is an only admin option | |
| 13:58:39 | tssurya | it shuld barf | |
| 13:58:42 | tssurya | should* | |
| 13:58:46 | tssurya | saying you are not admin | |
| 13:58:56 | mriedem | legacy backward compat | |
| 13:58:59 | tssurya | just lke tenant or user | |
| 13:59:03 | tssurya | ahhh ok ok | |
| 13:59:06 | mriedem | same reason you can do GET /servers?foo=bar | |
| 13:59:36 | mriedem | https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/servers.py#L606 | |
| 13:59:54 | openstackgerrit | Stephen Finucane proposed openstack/python-novaclient master: Microversion 2.68: Remove 'forced' live migrations, evacuations https://review.openstack.org/635131 | |
| 13:59:59 | mriedem | the query parameter schema is whitelisted but also allows users to pass anything for backward compat, | |
| 14:00:11 | mriedem | except the stuff in https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/servers.py#L512 | |
| 14:00:30 | mriedem | those are 404ed here https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L125 | |
| 14:01:34 | tssurya | mriedem: wow its crazy, but I understand at least why it was done, thanks for patiently explaining that | |
| 14:01:48 | tssurya | I'll work on ironing out the wrinkles | |
| 14:02:19 | mriedem | cool. otherwise from my manual testing (once i worked around those other bugs with the service version checking) it was working pretty well | |
| 14:02:57 | stephenfin | mriedem: Not sure you already discussed this again with kashyap, but any serious objections to doing the libvirt min version bump now? https://review.openstack.org/#/c/632507/ gibi and I are happy but I recall some objections previously | |
| 14:03:12 | kashyap | stephenfin: Heh, was _just_ about to write a message here. | |
| 14:03:26 | kashyap | stephenfin: Not quite objections, mriedem was not sure, based on some discussion w/ an operator he had | |
| 14:03:53 | kashyap | Ideally, I should have done this at the _start_ of the cycle, but completely lost track of it | |
| 14:04:24 | mriedem | looking, but you should probably ask the PTL | |
| 14:04:27 | adrianc | sean-k-mooney: Hi, regarding sriov live-migration patches, id like to address some of the comments in https://review.openstack.org/#/c/620115/21, however i saw you are planning to upload a new PS for the series, will it be soon ? as id like to avoid merge conflicts etc... | |
| 14:04:58 | kashyap | mriedem: Yeah, will do. I thought melwitt was still catching up after being on PTO (?) | |
| 14:05:28 | mriedem | stephenfin: kashyap: the concern from an operator was they were upgrading from mitaka to i think queens and the minimum bump in between was just for maintenance, not related to any functional changes, and the computes they were upgrading didn't have that minimum libvirt (they were using older centos i think) | |
| 14:06:01 | mriedem | so this operator was asking me if there was a reason for the bump, or if they could just revert the change to keep the older centos compute node but still upgrade nova | |
| 14:06:24 | sean-k-mooney | adrianc: i got pulled into some other stuff so go ahead. | |
| 14:06:28 | kashyap | I see. Yeah, probably we should better advertize _why_ we do bumps and at what intervals? | |
| 14:06:48 | kashyap | mriedem: ^ Like me writing a reminder note to the mailing list about why we do, and give a gentle heads-up. | |
| 14:06:55 | mriedem | i just think in years past we weren't real aggressive about doing it every release | |
| 14:07:09 | sean-k-mooney | adrianc: ill work on adressing the base patches but ill mainly be adressing comment/commit messages so there whould be no conflicts | |