Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-02-13
13:13:29 openstackgerrit Surya Seetharaman proposed openstack/nova master: API microversion 2.69: Handles Down Cells Documentation https://review.openstack.org/635147
13:15:22 openstackgerrit Jim Rollenhagen proposed openstack/nova master: ironic: partition compute services by conductor group https://review.openstack.org/635006
13:18:27 openstackgerrit Jim Rollenhagen proposed openstack/nova master: Ensure config regexes match the entire string https://review.openstack.org/636627
13:22:24 openstackgerrit Matt Riedemann proposed openstack/nova master: Change nova-next tempest test regex https://review.openstack.org/636459
13:28:12 Shilpa stephenfin: hi, under Taskflow, added https://bugs.launchpad.net/taskflow/+bug/1815738, do let me know is this active channel? Unable to trace PTL for this, but https://pypi.org/project/taskflow/ reports latest release in Jan'19
13:28:13 openstack Launchpad bug 1815738 in taskflow "Taskflow doesn't support to retrieve tasks progress details in the order of workflow task execution" [Undecided,New]
13:30:09 mriedem Shilpa: i think that would be in the #openstack-oslo channel now
13:31:11 mriedem or i guess #openstack-state-management https://wiki.openstack.org/wiki/TaskFlow#Contact_us.21
13:31:26 mriedem tssurya: did you enjoy my tales from going down cell down street?
13:31:53 Shilpa mriedem: #openstack-state-management here no one responding, will check #openstack-oslo channel
13:32:14 mriedem tssurya: belmoreira: i'm interested - what do you configure [upgrade_levels]/compute to in nova.conf? do you pin to a release or have it set to auto?
13:32:23 bauzas mriedem: FWIW, I uploaded the new rev for reshape
13:32:36 bauzas mriedem: I found a potential blocker, but eventually it's not a problem
13:32:55 mriedem ok cool
13:33:58 belmoreira mriedem: in the past used to pin it to a release. Now we use auto
13:36:56 openstackgerrit Lee Yarwood proposed openstack/nova master: Avoid redundant initialize_connection on source post live migration https://review.openstack.org/551302
13:37:37 mriedem belmoreira: ok if you have this change from stable/rocky https://review.openstack.org/#/c/624982/ then you might want to be careful,
13:37:43 mriedem i came across something nasty last night in down cell testing
13:38:02 mriedem https://bugs.launchpad.net/nova/+bug/1815697
13:38:03 openstack Launchpad bug 1815697 in OpenStack Compute (nova) "[upgrade_levels]compute=auto grinds the API response times when a cell is down" [Medium,Confirmed]
13:38:52 mriedem at this point i don't have any great solutions for that bug
13:40:00 sean-k-mooney mriedem: as a workaournd you set the retry interval to 1 and retries to 1 and set the exact version
13:40:29 sean-k-mooney that could lead to more api request failing in other cases but it should not be inherintly dangourous right
13:40:32 mriedem sean-k-mooney: no, the max_retries and retry_interval was just to quiet the logs
13:40:39 sean-k-mooney oh ok
13:40:46 mriedem the workaround for hanging the api on startpu was setting [upgrade_levels]/compute=rocky
13:42:11 tssurya mriedem: I did go through your tales :D I absolutely appreciate the testing :) I am adding more test cases for the parts you felt are lacking
13:42:51 belmoreira mriedem: we don't have that commit, yet. Thanks for the heads up.
13:42:53 tssurya and I am not so sure I get your worry here: https://review.openstack.org/#/c/591657/39/nova/api/openstack/compute/servers.py@135
13:43:38 Shilpa mriedem: no one replies here too #openstack-oslo, will wait for more time
13:45:41 mriedem Shilpa: ok well nova doesn't use taskflow. harlowja was the primary maintainer but i'm not sure if he's around anymore. oslo team would be your best bet i'd think, otherwise maybe #openstack-dev
13:45:58 mriedem efried loves taskflow though
13:46:49 Shilpa mriedem: ok, hope he will help me there
13:47:37 mriedem tssurya: my concern is if tooling is passing filter parameters by default,
13:47:50 mriedem normally in the API we filter out search options that the user isn't allowed to use without it being an error
13:48:18 mriedem so let's say i'm a non-admin and i pass some stuff which normally just gets filtered out in the remove_invalid_options method
13:48:25 mriedem essentially making search_opts = {}
13:48:41 tssurya like "deleted" and "tenant" ?
13:48:46 mriedem if i'm using 2.69, then empty search_opts would mean down_cell_support=True
13:48:59 mriedem tssurya: non-admins can't use the tenant filter
13:49:10 tssurya yea I know
13:49:12 mriedem servers are filtered by default from the project_id in the request context
13:49:27 tssurya which is why I was asking if that's what you mean by invalid options
13:50:04 mriedem so let's say i'm a non-admin and i'm using --deleted, here are some scenarios
13:50:25 mriedem 1. nova list --deleted with v2.1 - the deleted filter would be excluded w/o any errors in the api and i should list my servers
13:50:51 mriedem 2. nova list --deleted with v2.69 and list_records_by_skipping_down_cells=True (default), the down_cell_support flag would be set to False and i wouldn't get any results (if my servers are in a down cell)
13:51:06 mriedem 3. nova list --deleted with v2.69 and list_records_by_skipping_down_cells=False, if my servers are in a down cell i'll get a 500
13:51:45 mriedem my point was if we calculated the down_cell_support flag *after* calling remove_invalid_options to filter the search_opts, then #3 becomes #2
13:52:00 mriedem actually no that's not right,
13:52:14 mriedem i'd get partial results
13:53:14 openstackgerrit Lee Yarwood proposed openstack/nova master: Restore connection_info after live migration rollback https://review.openstack.org/551349
13:53:16 mriedem to summarize, i think the down_cell_support variable should be set based on the search opts after we've filtered that set in remove_invalid_options to increase the chances of someone getting partial results
13:53:17 lyarwood mdbooth: ^ btw, feel free to respin the commit message if you have time.
13:53:55 tssurya mriedem: ok, I get what you mean
13:54:21 mriedem the tricky thing is if you do that, you have to explicitly look for paging and sorting parameters
13:54:30 mriedem because of https://review.openstack.org/#/c/591657/39/nova/api/openstack/compute/servers.py@1235
13:54:38 tssurya yea
13:54:55 mriedem but i dont think that's so hard, we could just create a constant for ('sort_key', 'sort_dir', 'limit', 'marker') and use that in both places
13:55:34 mriedem and your _is_cell_down_supported method can check, "if any(PAGING_SORTING_PARAMS in reg.GET)
13:57:01 tssurya got it, let me dig more and incorporate that
13:57:17 mriedem err, if set(PAGING_SORTING_PARAMS) & set(list(req.GET.keys()))
13:57:20 mriedem something like that
13:57:33 mriedem does that make sense though?
13:57:45 tssurya it makes sense yea
13:57:59 openstackgerrit Hamdy Khader proposed openstack/os-vif master: Add create_ovs_port field in VIFPortProfileOpenVSwitch profile https://review.openstack.org/636061
13:58:01 tssurya but I still don't get why the policy things are mixed with ignoring flags
13:58:06 mriedem cool. my goal is to maximize the chance that someone gets partial results if they have servers in a down cell
13:58:29 mriedem which policy things?
13:58:37 tssurya I mean if --deleted is an only admin option
13:58:39 tssurya it shuld barf
13:58:42 tssurya should*
13:58:46 tssurya saying you are not admin
13:58:56 mriedem legacy backward compat
13:58:59 tssurya just lke tenant or user
13:59:03 tssurya ahhh ok ok
13:59:06 mriedem same reason you can do GET /servers?foo=bar
13:59:36 mriedem https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/servers.py#L606
13:59:54 openstackgerrit Stephen Finucane proposed openstack/python-novaclient master: Microversion 2.68: Remove 'forced' live migrations, evacuations https://review.openstack.org/635131
13:59:59 mriedem the query parameter schema is whitelisted but also allows users to pass anything for backward compat,
14:00:11 mriedem except the stuff in https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/schemas/servers.py#L512
14:00:30 mriedem those are 404ed here https://github.com/openstack/nova/blob/master/nova/api/openstack/compute/servers.py#L125
14:01:34 tssurya mriedem: wow its crazy, but I understand at least why it was done, thanks for patiently explaining that
14:01:48 tssurya I'll work on ironing out the wrinkles
14:02:19 mriedem cool. otherwise from my manual testing (once i worked around those other bugs with the service version checking) it was working pretty well
14:02:57 stephenfin mriedem: Not sure you already discussed this again with kashyap, but any serious objections to doing the libvirt min version bump now? https://review.openstack.org/#/c/632507/ gibi and I are happy but I recall some objections previously
14:03:12 kashyap stephenfin: Heh, was _just_ about to write a message here.
14:03:26 kashyap stephenfin: Not quite objections, mriedem was not sure, based on some discussion w/ an operator he had
14:03:53 kashyap Ideally, I should have done this at the _start_ of the cycle, but completely lost track of it
14:04:24 mriedem looking, but you should probably ask the PTL
14:04:27 adrianc sean-k-mooney: Hi, regarding sriov live-migration patches, id like to address some of the comments in https://review.openstack.org/#/c/620115/21, however i saw you are planning to upload a new PS for the series, will it be soon ? as id like to avoid merge conflicts etc...
14:04:58 kashyap mriedem: Yeah, will do. I thought melwitt was still catching up after being on PTO (?)
14:05:28 mriedem stephenfin: kashyap: the concern from an operator was they were upgrading from mitaka to i think queens and the minimum bump in between was just for maintenance, not related to any functional changes, and the computes they were upgrading didn't have that minimum libvirt (they were using older centos i think)
14:06:01 mriedem so this operator was asking me if there was a reason for the bump, or if they could just revert the change to keep the older centos compute node but still upgrade nova
14:06:24 sean-k-mooney adrianc: i got pulled into some other stuff so go ahead.
14:06:28 kashyap I see. Yeah, probably we should better advertize _why_ we do bumps and at what intervals?
14:06:48 kashyap mriedem: ^ Like me writing a reminder note to the mailing list about why we do, and give a gentle heads-up.
14:06:55 mriedem i just think in years past we weren't real aggressive about doing it every release
14:07:09 sean-k-mooney adrianc: ill work on adressing the base patches but ill mainly be adressing comment/commit messages so there whould be no conflicts
14:07:27 kashyap mriedem: Yes, I now kept a reminder on my phone to do the start of the cycle
14:07:33 adrianc sean-k-mooney: would you like me to address some of the nits in https://review.openstack.org/#/c/624842/ as well, or ill leave them to you ?

Earlier   Later