Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-04-03
13:32:35 sean-k-mooney its just a conflit in the tests
13:33:16 sean-k-mooney stephenfin: we might have some function that were added for the sriov migration code too
13:33:31 sean-k-mooney we merged all the resouce tracker code but have not merged the 2 patches that use it
13:33:49 openstackgerrit Matt Riedemann proposed openstack/nova stable/stein: Error out migration when confirm_resize fails https://review.openstack.org/649421
13:36:00 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove dead nova.db functions https://review.openstack.org/649570
13:36:21 mriedem sean-k-mooney: are you going to cherry pick https://review.openstack.org/#/c/649409/ to stable/stein now?
13:36:23 sean-k-mooney speaking of https://review.openstack.org/#/q/topic:bp/libvirt-neutron-sriov-livemigration+(status:open) if people have time to review that again it would be nice to see that merged now that train is open
13:36:27 mriedem efried: we're doing an rc2 for https://review.openstack.org/#/c/649409/ right?
13:36:50 gibi_off mriedem: I didn't find the specific reason for the warning in move_allocation so I guess I added it becase it thought that is should not really happen that we overwrite existing allocation during move
13:36:53 sean-k-mooney im waiting for zuul to merge it
13:37:08 sean-k-mooney but yes ill cherry pick it once that is done
13:37:28 mriedem gibi_off: it's there because of https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/migration-allocations.html
13:37:30 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove unused constants, functions https://review.openstack.org/649567
13:37:35 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove dead resource tracker code https://review.openstack.org/649569
13:37:41 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Remove dead nova.db functions https://review.openstack.org/649570
13:38:32 sean-k-mooney ... it going to fail again because fo failure in lower constarits
13:38:39 stephenfin mriedem, gibi_off: Have reshuffled those around to remove the placement changes and fix an issue with the last patch, FYI
13:38:52 sean-k-mooney http://logs.openstack.org/09/649409/3/check/openstack-tox-lower-constraints/1a8274d/testr_results.html.gz
13:40:13 mriedem i assume that's related to http://status.openstack.org/elastic-recheck/#1793364 but the signature on that query is old
13:40:22 gibi_off mriedem: so during revert resize/migrate the target allocation is non empty as we keep the allocation exists both on the source and the target host of the migration
13:41:38 mriedem during revert we should drop the target node allocations, held by the instance consumer, and move the source node allocations, held by the migratoin consumer, to the instance consumer
13:41:39 sean-k-mooney mriedem: well the patch to fix our lower constratins job just laned so maybe this is a bug that was fixed in later versions of sqlacamy/pymysql
13:41:54 mriedem so after revert there are no allocations for the instance on the target node and the instance has the source node allocations again
13:42:19 gibi_off mriedem: yeah I mixed up it is not moving allocation between hosts it moves between cosnumers
13:42:35 mriedem dansmith: sounds like we might be doing an RC2 for stein, and bug 1715374 is latent, but do you think it would be worth putting out a known issue release note for stein anyway?
13:42:36 openstack bug 1715374 in OpenStack Compute (nova) "Reloading compute with SIGHUP prevents instances from booting" [High,In progress] https://launchpad.net/bugs/1715374 - Assigned to Ralf Haferkamp (rhafer)
13:42:59 dansmith mriedem: I dunno, it's been this way for a long time apparently
13:43:06 dansmith so doesn't really seem like it
13:43:38 gibi_off mriedem: anyhow I think the warning can be deleted
13:43:53 mriedem gibi_off: or at least dropped to debug
13:44:03 gibi_off yeah
13:45:31 mriedem dansmith: just thinking about our upgrade docs and such that mention to use sighup during an upgrade https://docs.openstack.org/nova/stein/user/upgrade.html?highlight=sighup#concepts
13:45:51 dansmith well, I know
13:45:52 mriedem maybe that should have a note instead
13:46:01 dansmith yeah, that would make more sense I think
13:46:31 mriedem and https://docs.openstack.org/nova/stein/configuration/config.html?highlight=sighup#compute.resource_provider_association_refresh for efried
13:46:48 mriedem ^ might work for that option, but then kills the event listening stuff so you can't boot a server right?
13:46:56 mriedem neutron events i mean
13:48:21 mriedem blech we have stuff in here too https://docs.openstack.org/nova/stein/admin/configuration/schedulers.html?highlight=sighup#compute-capabilities-as-traits
13:49:59 sean-k-mooney so appraently rechecking a runing job does not work so i can either rebase to head of mater if peopel feel like +w again or i can recheck in 50mins when zuul finishies it current run
13:49:59 mriedem in other fun news, i learned last night that with cross-cell resize, we have to do a hard delete of the instance in a cell db rather than a soft delete
13:50:16 mriedem sean-k-mooney: just rebase
13:50:24 openstackgerrit sean mooney proposed openstack/nova master: Libvirt: gracefully handle non-nic VFs https://review.openstack.org/649409
13:51:20 sean-k-mooney done that will retriger the jobs and it just need +w. im going to grab lunch before a meeting so brb
13:54:40 mriedem so i wonder if we could recreate this sighup issue in one of our post-test hook scripts, we'd just sighup the local compute service and then create a server which should timeout waiting for the network-vif-plugged callback right?
13:54:53 dansmith no,
13:55:02 dansmith you have to have a server in the middle of creating when you sighup I think
13:55:07 dansmith before the event comes in
13:55:38 dansmith the other option is probably just to pick another signal as a stop-gap, register for it ourselves, and wire it up to our existing handlers
13:55:45 dansmith but that's even more icky for rc2
13:56:04 mriedem on a new server create after the signup, won't the _events dict be None and when registering the callback we'd hit this? https://review.openstack.org/#/c/420026/7/nova/compute/manager.py@316
13:56:53 dansmith I don't think so because we'll be re-created at the point before we get there
13:57:32 dansmith I think it only happens if we end up with a server in the middle of that while the event comes in, but once the sighup has finished the full restart, the new manager is hooked to new rpc connections, etc
13:57:38 dansmith it doesn't completely bork the server forever, AFAIK
13:57:48 dansmith else it would be really obvious that it's totally broken
13:58:10 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove unreachable codepaths https://review.openstack.org/649559
13:59:49 mriedem ok i thought the service was fubar after the sighup
13:59:57 mriedem so this is less severe than i thought
14:01:13 bauzas any urgent reviews I should do before RC2 tagging ?
14:01:33 dansmith not that I know of
14:01:36 trident mriedem: Regarding https://review.openstack.org/#/c/648653/
14:01:38 bauzas https://etherpad.openstack.org/p/nova-stein-rc-potential is pretty done
14:02:38 trident mriedem: You are correct, forbidden traits never worked unless there were also a required trait in the same flavor. If there were, both would be used, if not, the forbidden trait was lost.
14:03:14 bauzas mriedem: https://review.openstack.org/#/c/649454/ wanting it for RC2 ? I can vote on my own change given melwitt proposed it
14:03:40 bauzas I think it should clarify our Stein docs
14:03:49 bauzas even if we don't honestly branch them
14:06:04 mriedem bauzas: i'm waiting to hear that yes we're doing an rc2
14:06:15 mriedem and then i'd like to step through with the ptl which stein backports are going to go into it
14:06:28 bauzas soooooo... efried?
14:07:22 mriedem there is a 50% chance efried is the PTL coordinating the stein RC2 :)
14:08:54 bauzas oh, right, it's still melwitt's point :p
14:09:20 efried hi, sorry, catching up.
14:10:03 mriedem bauzas: you could start by backporting https://review.openstack.org/#/c/649409/ to stable/stein
14:10:36 bauzas mriedem: ok, I can do it
14:10:43 efried I think we need an RC2 for sure for https://review.openstack.org/#/c/649409/ at least
14:10:57 efried and if we're doing it, we might as well put the docs in, no risk there.
14:11:21 mriedem https://review.openstack.org/#/c/649454/ specifically yeah?
14:12:00 efried yes
14:13:11 efried mriedem: wanna +A that one please?
14:14:17 mriedem done
14:14:34 efried mriedem: Has anyone done docs on the SIGHUP thing?
14:14:40 efried L56
14:14:44 mriedem no
14:14:59 mriedem i was just creating a devstack env to try and recreate the bug
14:15:50 efried mriedem: I can go through and add a quick
14:15:50 efried .. note:: SIGHUP is broken, see `bug 1715374`_
14:15:52 openstack bug 1715374 in OpenStack Compute (nova) "Reloading compute with SIGHUP prevents instances from booting" [High,In progress] https://launchpad.net/bugs/1715374 - Assigned to Ralf Haferkamp (rhafer)
14:15:52 efried to all the docs where it's mentioned if you like.
14:16:57 mriedem well, if the nuance is it's broken for a window while servers are being created and waiting for an event when the sighup runs, that gets a bit hard to communicate in all the places we have sighup mentioned in the docs
14:17:36 efried .. note:: SIGHUP behavior is questionable, see `bug 1715374`_ if you try it and things go wobbly.
14:17:37 openstack bug 1715374 in OpenStack Compute (nova) "Reloading compute with SIGHUP prevents instances from booting" [High,In progress] https://launchpad.net/bugs/1715374 - Assigned to Ralf Haferkamp (rhafer)
14:17:37 efried ?
14:18:28 mriedem idk, could give a more detailed explanation of the bug in the reset() portion of https://docs.openstack.org/nova/latest/reference/services.html#the-nova-manager-module for just the compute service
14:19:06 mriedem i wouldn't want to mention a bug in 5 places just to have to remember after the bug is fixed to go back and remove all of those 5 places
14:20:18 efried as you wish
14:21:00 openstackgerrit Sylvain Bauza proposed openstack/nova stable/stein: Libvirt: gracefully handle non-nic VFs https://review.openstack.org/649630
14:21:09 bauzas mriedem: ^
14:27:36 mriedem is there anything else we want to consider? https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/stein
14:29:12 bauzas mriedem: https://review.openstack.org/#/c/647310/ is controversial for RC2
14:31:24 dansmith I thought we said no to that?

Earlier   Later