| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-20 | |||
| 18:15:51 | sean-k-mooney | artom: this can happen for example if you did and inplace upgrade and then resize the vm | |
| 18:16:09 | artom | sean-k-mooney, aha, ok | |
| 18:16:11 | sean-k-mooney | normally the changes would get picked up on the next hard reboot | |
| 18:16:56 | sean-k-mooney | artom: but ya were you actully hittin an issue? | |
| 18:17:10 | artom | sean-k-mooney, not 100% sure yet, but looks like it's Neutron | |
| 18:17:23 | artom | We ask for a thing, and timeout waiting for it | |
| 18:18:16 | artom | Grr, and I can't find the corresponding Neutron request | |
| 18:18:35 | sean-k-mooney | oh is this the virtual interface thing we were talking about downstream earlier | |
| 18:19:48 | artom | Yeah | |
| 18:20:29 | sean-k-mooney | artom: as i mentioned to mdbooth eariler if any of the storage cleanup code raises an exception in _rollback_live_migration its possible today that we will never call self.network_api.setup_network_on_host to rollback the network | |
| 18:20:51 | sean-k-mooney | so if there is an exption earilier in the function https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L6958 will not be called | |
| 18:21:01 | artom | sean-k-mooney, no, I think we're reaching that part | |
| 18:21:10 | artom | As I said, I see Nova making the Neutron request | |
| 18:21:39 | artom | But debug's not enabled, so I can't find that request arriving on the Neutron side | |
| 18:21:54 | artom | I mean, maybe it doesn't? | |
| 18:22:09 | sean-k-mooney | you shoudl see it in the neutron api log without debugging | |
| 18:22:10 | artom | Actually, debug *is* enabled | |
| 18:22:58 | sean-k-mooney | you may not see al of the intermediate steps but the post to the port binding endoint should be there | |
| 18:23:32 | artom | It'll log the HTTP method, right? | |
| 18:23:45 | artom | I'm casting a wide net and straight up grep -E '2019-02-14 18:50:5.*(PUT|POST)' -R `find . -wholename '*neutron/*.log'` | |
| 18:23:46 | sean-k-mooney | ya and it should have the respocne code | |
| 18:23:51 | artom | Nothing. | |
| 18:25:23 | artom | That's weird though, 'cuz Nova would log a timeout | |
| 18:25:44 | sean-k-mooney | so it timed out waiting for the network-vif-plugged event | |
| 18:26:12 | sean-k-mooney | whcich got raised as a VirtualInterfaceCreateExcetiopn. | |
| 18:27:00 | sean-k-mooney | the logs i have did not containt the neutron server logs | |
| 18:27:30 | artom | sean-k-mooney, that might be why | |
| 18:46:52 | imacdonn | Is it just me, or does this releasenote seem incomprehensive? I'm left wondering whether or not I should remove the consoleauth service, and under what circumstances the workaround is required. "A new check is added to the nova-status upgrade check CLI to check for use of the nova-consoleauth service to warn and provide additional instructions to set [workarounds]enable_consoleauth = True while performing a live/rolling upgrade." | |
| 18:59:41 | imacdonn | I think it should say something like "The consoleauth service is deprecated, and should be disabled/removed, however, if there is a requirement to maintain support for existing console sessions through a live/rolling upgrade......" | |
| 19:00:55 | melwitt | consoleauth strikes again | |
| 19:01:30 | melwitt | imacdonn: thanks for the feedback. I actually tried to explain it better in the latest updates to the releasenote and still missed the mark. I can propose another change | |
| 19:01:57 | melwitt | you definitely need to enable the workaround if you're doing a live upgrade | |
| 19:02:27 | imacdonn | yeah, seems we've been here before ;) At least removing the service seems to actually not generally break things this time ;) | |
| 19:02:35 | aspiers | melwitt: are release notes only for operator- / user-facing changes? | |
| 19:02:54 | aspiers | just wondering if I should add one for this new driver-capabilities-to-traits patch | |
| 19:06:42 | imacdonn | heh | |
| 19:07:07 | melwitt | aspiers: yeah, anything users should know about a change. I don't think I've seen the driver capabilities to traits thing yet | |
| 19:07:29 | aspiers | melwitt: https://review.openstack.org/#/c/538498/12 | |
| 19:07:40 | aspiers | melwitt: it's my rebase of an old patch from mriedem | |
| 19:07:45 | aspiers | I'm currently fixing the test failures | |
| 19:09:31 | imacdonn | aspiers: I guess my question (if I mattered) would be: who needs to know about this ? | |
| 19:09:45 | melwitt | aspiers: ok, I see. I think that would use a release note to let users know that driver capabilities will show up as custom traits in the placement API | |
| 19:10:18 | aspiers | imacdonn: my question too ;-) | |
| 19:10:24 | aspiers | melwitt: OK thanks! | |
| 19:12:26 | melwitt | aspiers: as for who, I don't know exactly, but it's the kind of thing I could see a cloud monitoring system looking at, what driver capabilities exist on each compute host | |
| 19:12:51 | aspiers | melwitt: yup, makes sense | |
| 19:13:22 | imacdonn | if there's a change to the way things appear through any published interface, I'd say it definitely should be a release-note. | |
| 19:13:52 | aspiers | yeah, I suppose just new traits appearing would be sufficient to justify documenting | |
| 19:18:15 | imacdonn | tangentially related (to release notes), I found a typo in another one. I might submit a fix a bit later, if I get time, but if someone else wants to grab it, that'd be cool too. https://bugs.launchpad.net/nova/+bug/1816831 | |
| 19:18:15 | openstack | Launchpad bug 1816831 in OpenStack Compute (nova) "DOC: typo in add_initial_allocation_ratio releasenote" [Undecided,New] | |
| 19:20:52 | melwitt | thanks | |
| 19:33:23 | mriedem | dansmith: +2 on that docs change | |
| 19:33:58 | dansmith | mriedem: +W | |
| 19:34:05 | mriedem | let us never speak of this again | |
| 19:34:25 | dansmith | I looked a the best practices one a bit, found one typo, but will look again when it gets respun | |
| 19:35:42 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Fix typo in initial_disk_allocation_ratio release note https://review.openstack.org/638245 | |
| 19:35:44 | mriedem | imacdonn: done - thanks for reporting that ^ | |
| 19:36:10 | imacdonn | mriedem: thanks! | |
| 19:37:12 | dansmith | +W within one minute | |
| 19:37:46 | melwitt | whew, what a sense of accomplishment | |
| 19:39:28 | aspiers | almost as towering an achievement as https://review.openstack.org/#/c/609777/ | |
| 19:39:45 | melwitt | lol | |
| 19:39:49 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: Add support for microversion 2.70 - expose device tags https://review.openstack.org/636779 | |
| 19:39:54 | aspiers | melwitt: my finest hour | |
| 19:46:04 | artom | aspiers, I'm grateful that your around | |
| 19:48:23 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Replace ansible --sudo with --become in live_migration/hooks scripts https://review.openstack.org/635308 | |
| 19:52:41 | imacdonn | artom: I see what you did there ;) | |
| 20:02:27 | artom | imacdonn, I no you did. | |
| 20:02:36 | imacdonn | :) | |
| 20:03:12 | aspiers | artom: stop teasing, its cruel | |
| 20:18:01 | aspiers | ;) | |
| 20:52:41 | mriedem | low-hanging-fruit docs bug if someone wants it https://bugs.launchpad.net/nova/+bug/1816859 | |
| 20:52:42 | openstack | Launchpad bug 1816859 in OpenStack Compute (nova) "Server concepts in nova - automatic resize confirm is wrong in docs" [Medium,Triaged] | |
| 20:54:30 | openstackgerrit | Adam Spiers proposed openstack/nova master: Convert driver supported capabilities to compute node provider traits https://review.openstack.org/538498 | |
| 20:59:09 | openstackgerrit | Adam Spiers proposed openstack/nova master: Convert driver supported capabilities to compute node provider traits https://review.openstack.org/538498 | |
| 21:00:01 | aspiers | efried_pto, mriedem, melwitt: I think this is ready for serious consideration now ^^^ | |
| 21:00:24 | aspiers | I've fixed all the TODOs I can see, and (fingers crossed) the tests will pass | |
| 21:00:35 | aspiers | s/will/should/ | |
| 21:04:04 | melwitt | kewl | |
| 21:08:24 | mriedem | aspiers: so is your sev stuff going to rebase on top of that or what? | |
| 21:09:09 | mriedem | because it refers to both a blueprint that's already complete and one that's not approved | |
| 21:15:19 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize while deleting a server https://review.openstack.org/638268 | |
| 21:15:19 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Add cross-cell resize policy rule and enable in API https://review.openstack.org/638269 | |
| 21:15:26 | mriedem | dansmith: ^ that's the end of the road, let's ship it | |
| 21:34:33 | openstackgerrit | Sundar Nadathur proposed openstack/nova master: ksa auth conf and client for cyborg access https://review.openstack.org/631242 | |
| 21:36:04 | artom | mriedem, heads up, we're hitting something similar to https://bugs.launchpad.net/nova/+bug/1813789 in our CI, except with revert_resize rather than evacuate | |
| 21:36:05 | openstack | Launchpad bug 1813789 in OpenStack Compute (nova) "Evacuate test intermittently fails with network-vif-plugged timeout exception" [Medium,Confirmed] | |
| 21:36:16 | artom | Ie, listening too late for an event | |
| 21:36:27 | artom | So looks like I'm fixing both of those | |
| 21:37:01 | artom | Would it make sense to file a different bug? Or just append to that one? | |
| 21:37:31 | sean-k-mooney | i would just use ^ | |
| 21:37:52 | mriedem | artom: that bug is f'ed | |
| 21:38:03 | mriedem | i think we *are* registered to wait for the event | |
| 21:38:05 | mriedem | and neutron isn't sending it | |
| 21:38:34 | sean-k-mooney | mriedem: in the down stream bug neutron is sending it before we register to recive it | |
| 21:38:48 | mriedem | need to figure out what is triggering the event then | |
| 21:38:50 | mriedem | because for evacuate, | |
| 21:38:54 | artom | mriedem, and that's what you describe in your bug as well | |
| 21:39:00 | mriedem | the driver.spawn() is registering the callback | |
| 21:39:14 | mriedem | what might be sending the event is the port binding change | |