Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-10-29
19:56:57 mnaser ah shit
19:57:42 mnaser ill run that locally
19:58:03 mriedem +2 on the top one
20:21:12 openstackgerrit Mohammed Naser proposed openstack/nova master: libvirt: Avoid setting MTU during live migration if unset https://review.openstack.org/614008
20:21:12 openstackgerrit Mohammed Naser proposed openstack/nova master: Add tests for bug #1800511 https://review.openstack.org/614004
20:21:13 openstack bug 1800511 in OpenStack Compute (nova) "VMs with vif_type bridge/tap started before Rocky upgrade cannot be live migrated" [High,In progress] https://launchpad.net/bugs/1800511 - Assigned to Mohammed Naser (mnaser)
20:21:30 mnaser mriedem: fixed your nit too if you feel like readding your +2 and cfriesen +1 :)
20:24:28 mriedem +Ws
20:24:30 mriedem oops
20:24:32 mriedem +2s
20:24:46 mriedem time to play the "find another core" game
20:36:22 mnaser mriedem: I guess all the IBM ones are busy today :p
20:38:34 mriedem too soon
20:42:25 mnaser It’s been quiet here today
20:42:41 mnaser And it’s getting late so I might have to settle for an EU core tomorrow :p
20:45:47 melwitt I can take a look. things have been a bit chaotic today
20:48:28 mriedem the one day sean-k-mooney isn't working 20 hours
20:49:16 mriedem mnaser: you've deployed this patch now and verified it fixes your upgrade issue?
20:49:30 melwitt do we want sean-k-mooney to test this first?
20:49:42 mriedem i think mnaser is testing it in prod
20:49:56 mnaser I have not tested in the environment to be honest
20:50:08 mnaser I can do that now tho
20:50:15 mnaser Gimme a few
20:50:28 dansmith seems pretty clear from the bug that this is the right thing to do
20:50:38 melwitt ok, asked in case that's why you mentioned sean-k-mooney
20:50:50 dansmith obviously a manual test would be nice
20:51:06 dansmith I dropped a +W on the top meaty one.. I can put an upgradeable +1 on the bottom one
21:03:25 mnaser (taking a bit of time, trying to find VMs that have been launched before which are testing)
21:14:33 mnaser dansmith: mriedem melwitt i've tested this one now locally on a vm that confirmed had no mtu set (virsh dumpxml, /proc/pid/cmdline had no host_mtu), live migrated with patch and migration completed succesfully
21:18:52 mriedem i mentioned sean-k-mooney because he added the mtu stuff for bridge/tap types
21:19:04 mriedem and is familiar with this live migration _update_vif_xml stuff
21:20:48 melwitt ack
21:23:51 mriedem while we're all here and having fun, we need to do a rocky release for several upgrade related issues, but need https://review.openstack.org/#/q/topic:bug/1798188+(status:open+OR+status:merged) as part of that,
21:23:58 mriedem and will also want the fix that mnaser just pushed
21:24:02 mriedem mnaser: are you working backports?
21:24:18 efried mriedem: Allocation ratio fix will need to be part of --^ ?
21:24:32 mriedem efried: i don't think that is critical
21:24:36 mriedem annoying, yes
21:24:39 efried okay
21:24:39 mriedem but not critical
21:28:12 melwitt mriedem: I was debating how to update the reno as mentioned by mnaser on the review
21:29:18 melwitt because once all compute hosts are rocky, should be OK to stop running nova-consoleauth with the workaround BUT you also have to make sure you deploy your console proxies per cell and you will still experience all consoles being reset that the point you stop using the workaround (if not all auths already expired)
21:29:58 melwitt it's kind of complicated so I was feeling stuck on whether the reno is conveying all of that well enough
21:31:21 mriedem well, it says use the workaround while you're doing rolling upgrade, so not all computes are rocky yet, as mohammed pointed out,
21:31:51 mriedem and then there is the "Operators may unset the configuration option when:" section which says once you're upgraded (all hosts are Rocky), and stuff is expired, you can drop the workaround,
21:32:01 mriedem if you need to add "and you're running console proxy per cell" to that list, then just add that
21:32:38 melwitt yeah, ok. thanks. I've felt so mired in it that I was having a hard time thinking about it
21:32:49 mriedem i left comments,
21:33:17 mriedem but i think it's just (1) describing what rolling upgrade means and (2) add the part about console proxy being deployer per cell before unsetting the workaround
21:34:02 mriedem we have checks for the former in the nova-status upgrade check, but not the latter, but i'm not sure if it's possible to check the latter via nova-status is it?
21:39:46 openstackgerrit Matt Riedemann proposed openstack/nova master: Add Destination.cross_cell_move field https://review.openstack.org/614035
21:39:46 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Cross-cell resize https://review.openstack.org/603930
21:39:47 openstackgerrit Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.openstack.org/614037
21:39:47 openstackgerrit Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.openstack.org/614036
21:39:57 melwitt mriedem: I don't think so. the only possible way to check is if the nova.conf on the console proxy host has [database]connection configured (implying access to a cell database)
21:42:08 mnaser mriedem: sorry checking things in and out, let me push up backport
21:42:28 mnaser I wonder if the UI adds the cherry picked from if it hasn’t merged yet
21:42:32 mriedem it does not
21:42:46 mnaser Boo. Alright
21:42:52 mnaser This needs to go back to rocky or further?
21:43:05 mriedem_away rocky
21:43:09 mnaser ok cool
21:43:36 mriedem_away _update_vif_xml was added in rocky,
21:44:10 mriedem_away setting the mtu was backported to queens. if that could fail for different reasons, like live migrating a guest from source host which set mtu to something and then to a dest host that is not new enough to support that mtu config, well that's a different issue
21:44:45 mnaser mriedem_away: most likely not, this feature was added a while back ago
21:44:51 mnaser let me verify that
21:45:01 mriedem_away libvirt 3.3
21:45:27 mnaser oh yes only that got support for it
21:45:39 mriedem_away but our minimum libvirt support is 1.3.1
21:45:56 mnaser so unless someone is running 1.3.1 alongside 3.3.0
21:46:08 mriedem_away i think this might catch it anyway https://github.com/openstack/nova/blob/master/nova/conductor/tasks/live_migrate.py#L235
21:46:37 mriedem_away alright really leaving now
21:46:42 openstackgerrit Mohammed Naser proposed openstack/nova stable/rocky: libvirt: Avoid setting MTU during live migration if unset https://review.openstack.org/614041
21:46:42 openstack bug 1800511 in OpenStack Compute (nova) "VMs with vif_type bridge/tap started before Rocky upgrade cannot be live migrated" [High,In progress] https://launchpad.net/bugs/1800511 - Assigned to Mohammed Naser (mnaser)
21:46:42 openstackgerrit Mohammed Naser proposed openstack/nova stable/rocky: Add tests for bug #1800511 https://review.openstack.org/614040
21:46:43 mnaser ^ thats for later
22:08:07 openstackgerrit Eric Fried proposed openstack/nova master: Kill @safe_connect in _get_provider_traits https://review.openstack.org/613613
22:10:00 openstackgerrit Eric Fried proposed openstack/nova master: Kill @safe_connect in _get_provider_traits https://review.openstack.org/613613
23:11:21 openstackgerrit melanie witt proposed openstack/nova stable/rocky: Add rolling upgrade info to enable_consoleauth workaround reno https://review.openstack.org/613174
23:12:25 melwitt mnaser: updated to address your comments ^ let me know if there's anything else unclear in the message
23:14:36 openstackgerrit melanie witt proposed openstack/nova stable/rocky: Add rolling upgrade info to enable_consoleauth workaround reno https://review.openstack.org/613174
#openstack-nova - 2018-10-30
00:15:31 openstackgerrit Matt Riedemann proposed openstack/nova master: Add Destination.cross_cell_move field https://review.openstack.org/614035
00:15:31 openstackgerrit Matt Riedemann proposed openstack/nova master: Add Migration.cross_cell_move and get_by_uuid https://review.openstack.org/614012
00:15:32 openstackgerrit Matt Riedemann proposed openstack/nova master: Change HostManager to allow scheduling to other cells https://review.openstack.org/614037
00:15:32 openstackgerrit Matt Riedemann proposed openstack/nova master: Add InstanceAction/Event create() method https://review.openstack.org/614036
00:15:33 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: Cross-cell resize https://review.openstack.org/603930
01:29:27 openstackgerrit Chason Chan proposed openstack/nova stable/rocky: Remove deprecated nova-consoleauth reference from doc https://review.openstack.org/614055
01:57:59 openstackgerrit Takashi NATSUME proposed openstack/nova master: Use links to placement docs in nova docs https://review.openstack.org/614056
05:12:27 openstackgerrit Merged openstack/nova master: Drop legacy live migrate allocation compat code https://review.openstack.org/611975
07:31:09 openstackgerrit Merged openstack/nova master: Fix ironic client ironic_url deprecation warning https://review.openstack.org/611872
08:16:50 bauzas good morning Nova
08:45:38 kashyap mnaser: Hey, I saw the ping here. But I guess your live migration issue is nailed down now, thanks to mdbooth_390 & co
08:55:08 mnaser kashyap: indeed, your comment on that bz helped point me towards the why :)
08:55:14 mnaser So thanks! :)
08:55:45 kashyap mnaser: Which bugzilla?
08:55:57 kashyap Sorry, just catching up after 10 days of travel and conferences
08:56:33 mnaser kashyap: https://bugzilla.redhat.com/show_bug.cgi?id=1449346
08:56:34 openstack bugzilla.redhat.com bug 1449346 in libvirt "Addition of host_mtu=XXXX changes PCI ioport size & addresses for virtio-net device" [Unspecified,Closed: errata] - Assigned to laine
08:56:45 mnaser Which referenced in https://review.openstack.org/#/c/614008/
08:56:51 kashyap Ah, that one.

Earlier   Later