Earlier  
Posted Nick Remark
#openstack-nova - 2020-11-09
14:23:31 lyarwood yet*
14:24:18 gibi sweet
14:26:58 gibi I guess then we can check our docs and replace the nova CLI exemples with openstack CLI examples: http://paste.openstack.org/show/799826/
14:27:27 gibi bahh that paste contains duplicates
14:28:08 gibi nah, much better now http://paste.openstack.org/show/799828/
14:30:04 stephenfin sounds reasonable; I could probably do that this week
14:30:19 openstackgerrit Balazs Gibizer proposed openstack/nova master: Use the non polling notification waiter in func test https://review.opendev.org/758445
14:30:19 openstackgerrit Balazs Gibizer proposed openstack/nova master: Create a fixture around fake_notifier https://review.opendev.org/758446
14:30:20 openstackgerrit Balazs Gibizer proposed openstack/nova master: Use NotificationFixture for legacy notifications too https://review.opendev.org/758448
14:31:45 openstackgerrit Balazs Gibizer proposed openstack/nova master: Test the NotificationFixture https://review.opendev.org/758450
14:31:45 openstackgerrit Balazs Gibizer proposed openstack/nova master: Move fake_notifier impl under NotificationFixture https://review.opendev.org/758451
14:31:50 gibi stephenfin: cool. you can plug me in for review
14:32:25 gibi I think the (only) agreement on the PTG about OSC was that we try to fix our docs to use OSC examples
14:33:00 brinzhang_ stephenfin: +1, good start
14:34:13 sean-k-mooney gibi: once we have parity however i really would like to look at deprecating novaclient and requireing osc to be updated
14:34:35 sean-k-mooney we can defer that for a cycle or so but i would like that to be our medium term goal
14:34:44 sean-k-mooney even if we dont do it this cycle
14:34:53 stephenfin sean-k-mooney: that's happening already
14:35:04 gibi sean-k-mooney: agree
14:35:04 sean-k-mooney not quite
14:35:16 stephenfin gtema is doing a lot of work here
14:35:25 sean-k-mooney stephenfin: wehn i say deprecate novaclint i mean stop accpeting any new features/commands being added
14:35:50 sean-k-mooney so its bugfix only and require that they are added to osc instead
14:36:18 stephenfin to the CLI, sure, but we should keep adding to the library code until we're sure openstacksdk is good enough
14:36:25 sean-k-mooney preferably doing one final realse of novaclinet when we freeze the features
14:36:47 sean-k-mooney stephenfin: maybe i was not making that distintion really
14:36:52 stephenfin I'd like to move pretty much all of OSC first, since we should have no issues doing that once openstacksdk is good enough
14:37:54 sean-k-mooney since you have put this effort in to close the gap i just dont want it to reopen
14:44:14 openstackgerrit sean mooney proposed openstack/nova master: libvirt: delegate ovs plug to os-vif https://review.opendev.org/602432
14:44:48 sean-k-mooney stephenfin: lyarwood can ye take a look at ^
14:44:58 sean-k-mooney fixed the pep8 issue otherwise its the same
14:47:31 openstackgerrit Balazs Gibizer proposed openstack/nova master: doc: require openstack client change for every new API microversion https://review.opendev.org/717727
14:47:54 gibi stephenfin, sean-k-mooney: this ^^ is my contribution to the OSC topic
14:49:21 sean-k-mooney :)
14:51:50 gibi "Delay in Elastic Search: Indexing behind by 141 hours" :(
14:54:50 sean-k-mooney thats only slightly longer then normally its normally 72 hours i think
14:55:26 sean-k-mooney if you are fering to logstash/kibana upstream
14:55:32 sean-k-mooney *refering
15:01:35 gibi it remember when it was close to 0
15:01:41 gibi last week it hanged around 100 hours
15:02:34 sean-k-mooney it does tend to vary but i think the target was no more the 72 hours
15:02:45 sean-k-mooney it does catch up from time to time
15:03:45 gibi I hope so
15:13:12 brinzhang_ stephenfin: thanks fix that issue from openstack server migration list CLI, I am sure I was tested ti, but the strange thing is that this issue was not found
15:21:05 openstackgerrit Balazs Gibizer proposed openstack/nova master: Remove compute service level check for qos ops https://review.opendev.org/735570
15:21:52 gibi stephenfin: I think you will like this code removal patch ^
15:32:38 openstackgerrit Balazs Gibizer proposed openstack/nova stable/victoria: Warn when starting services with older than N-1 computes https://review.opendev.org/761923
15:42:10 openstackgerrit Balazs Gibizer proposed openstack/nova stable/victoria: Add upgrade check about old computes https://review.opendev.org/761924
15:45:17 stephenfin gibi: Done. I assume we're okay to merge things like that now that we've got the service version check?
15:45:31 gibi stephenfin: that is my idea too
15:45:56 dansmith the per-release service version check should catch anything that merged N-2 releases ago,
15:46:11 dansmith so it should be fine to remove older ones and rely on the macro one yeah
15:46:50 dansmith (assuming the qos one is old enough, I didn't look)
15:47:05 gibi qos move ops are added in Ussuri
15:47:35 gibi I mean that last one
15:48:15 gibi so in Victoria we could have removed the service level check. But we did not for extra safety
15:49:48 openstackgerrit Lee Yarwood proposed openstack/nova master: Migrate nova-grenade-multinode job to zuulv3 native https://review.opendev.org/742056
15:52:22 dansmith stephenfin: we're not really breaking RPC specifically here, because we're not changing any rpc versions or signatures or anything, but this is one of those "not covered by the rpc versions" behaviors.. it's breaking service-to-service interaction, but only for older computes (not even older RPC versions), but for which we've already said isn't supported
15:53:03 stephenfin that makes sense
15:53:05 lyarwood gmann: ^ I'd like to push ahead with this btw, we are currently hitting https://bugs.launchpad.net/nova/+bug/1901739 with the original bionic based job so I'd rather switch to Focal and add the ceph coverage later.
15:53:05 openstack Launchpad bug 1901739 in OpenStack Compute (nova) " libvirt.libvirtError: internal error: missing block job data for disk 'vda'" [High,Fix released] - Assigned to Lee Yarwood (lyarwood)
15:53:08 dansmith stephenfin: I'll put this in a comment once I review, but just echoing here since you called me out :)
15:53:18 stephenfin ta :
15:53:19 stephenfin :)
15:53:37 gibi stephenfin, dansmith: thanks for the review btw
16:05:08 openstackgerrit Merged openstack/nova master: Remove six.moves https://review.opendev.org/727224
16:10:06 gmann lyarwood: but we are going to loose ceph coverage right? or we can add ceph coverage as separate job using existing script and move them once ceph greande base job is ready
16:10:35 lyarwood gmann: we can try but I'd take a working gate over missing ceph coverage for a few weeks at the moment
16:12:32 gmann lyarwood: existing zuulv2 grenade jobs is working right or it is failing?
16:12:34 jgwentworth lyarwood: so what's the plan for adding ceph coverage back? seems like a risk to leave it uncovered for an extended period of time. do we have any idea how to do it for v3 jobs?
16:13:18 lyarwood gmann: it's failing pretty often at the moment due to a an issue with libvirt/QEMU on bionic
16:13:56 gmann you mean on victoria gate? on master gate, it should run on Focal
16:13:56 lyarwood melwitt: https://review.opendev.org/#/q/owner:self+topic:native-zuulv3-migration+status:open - wire up a native zuulv3 multinode ceph job
16:14:13 lyarwood gmann: master gate, multinode grenade still uses bionic
16:14:23 lyarwood gmann: and that's the problem here
16:14:32 lyarwood gmann: or we can move it to NV
16:14:41 gmann lyarwood: oh we should move it to Focal
16:15:04 gmann ah legacy job
16:15:09 melwitt lyarwood: sorry, I don't understand what I'm looking at here that's related to ceph
16:15:41 gmann lyarwood: i did not move base legacy job on bionic
16:17:47 gmann lyarwood: I think we can move to zuulv3 using script for now like in PS1 - https://review.opendev.org/#/c/742056/1 and once ceph grenade base is ready then remove the use of script ?
16:18:06 lyarwood gmann: we can try
16:18:24 gmann ok let me update it.
16:22:03 lyarwood melwitt: sorry nova-grenade-multinode currently does some ceph stuff manually via the live migration hook - https://github.com/openstack/nova/blob/80b807a4c590980b9e514042778bd8c277e89e40/playbooks/legacy/nova-grenade-multinode/run.yaml#L58 & https://github.com/openstack/nova/blob/80b807a4c590980b9e514042778bd8c277e89e40/gate/live_migration/hooks/run_tests.sh#L55-L65
16:22:41 lyarwood melwitt: I wanted to break this out into a native zuulv3 job based on a multinode ceph job but that's taking a while to work out on the topic I shared above
16:23:37 melwitt lyarwood: ok, so the manual stuff could be made "native" somehow. I did not know that
16:24:25 lyarwood melwitt: yeah instead of calling specific bash functions from the plugin I just wanted to have a generic job that would deploy multinode ceph that we'd run the LM tests on in Nova
16:25:00 lyarwood melwitt: I made some progress a while ago with the key sharing etc just became stuck at the end with getting the subnode to actually connect to the ceph cluster on the main node
16:26:23 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Add live migration tests for PCI, SR-IOV servers https://review.opendev.org/746950
16:26:23 openstackgerrit Stephen Finucane proposed openstack/nova master: functional: Expand SR-IOV live migration tests with NUMA https://review.opendev.org/749360
16:27:12 melwitt lyarwood: I see, thanks, that helps. I hoped to be able to help in some way since I am concerned about the coverage loss but didn't know where to look or start
16:32:17 lyarwood gmann: so are you just going to change the base job and hope that works?
16:32:46 lyarwood to tempest-multinode-full-py3
16:34:14 openstackgerrit Merged openstack/nova master: Remove six.iteritems/itervalues/iterkeys https://review.opendev.org/727757
16:34:25 openstackgerrit Merged openstack/nova master: Remove six.byte2int/int2byte https://review.opendev.org/727777
16:35:24 gmann lyarwood: not tempest but grenade-multinode and running run_tests.sh in post phase with disable to run smoke tests on new node (which run as part of grenade-multinode playbooks)
16:37:17 lyarwood gmann: right I'm asking which base job you're going to use
16:37:53 gmann lyarwood: for grenade anyways we need to use grenade-multinode
16:39:21 lyarwood gmann: right sorry, that's zuulv3 based and would just use the scripts to avoid us losing coverage
16:39:48 gmann yeah

Earlier   Later