Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-01-25
22:33:02 melwitt that is, I didn't expect the events to be ignored when they're being waited for
22:36:06 mriedem melwitt: it's not saying it's ignored b/c of the instance state
22:36:20 mriedem it's saying the compute got the event but no callback was registered waiting for it
22:36:28 mriedem Jan 19 10:33:35.986648 ubuntu-xenial-rax-iad-0002010825 nova-compute[28907]: DEBUG nova.compute.manager [None req-4bcf94cf-2048-4e99-94e6-646519900469 service nova] [instance: dd2ec3b0-f909-4104-ae2a-9b878d936ed4] No waiting events found dispatching network-vif-plugged-567525a4-4b16-4004-91d5-545086d01227 {{(pid=28907) pop_instance_event /opt/stack/new/nova/nova/compute/manager.py:363}}
22:37:49 melwitt mriedem: okay, I was going by the last comment in the bug. okay, so maybe there's a bug in the _create_domain_and_network vifs_already_plugged logic somewhere
22:38:29 melwitt or maybe vifs_already_plugged doesn't mean what I thought it means
22:40:45 mriedem i wonder if we don't register the callback before the vif plug event has already happened
22:41:10 mriedem no that doesn't make sense
22:41:15 mriedem we register the callback before calling plug_vifs
22:41:26 melwitt "Timeout waiting for vif plugging callback for instance with vm_state active and task_state reboot_started_hard.: Timeout: 300 seconds" so that means it was waiting
22:45:02 melwitt so "Received unexpected event" happens when there weren't events in self.instance_events in compute manager. so how do they get there ...
22:45:07 dansmith mriedem: yeah the whole design of that thing is so you can't trigger before you're registered
22:56:18 mriedem i don't know what to do about this bug https://bugs.launchpad.net/nova/+bug/1740123
22:56:20 openstack Launchpad bug 1740123 in OpenStack Compute (nova) "Can we create server group with empty policies?" [Undecided,New]
22:56:25 mriedem it's not a bug per se
22:56:51 melwitt mriedem: it seems like the patch is related to the new bug. I think the only way this can happen is if _get_neutron_events(network_info) is returning no events, meaning there are no vifs in network_info. I'm looking to see where/how does network_info get reset after a reboot
22:57:34 mriedem melwitt: if it doesn't return any events, wouldn't we just not wait for anything?
22:57:39 mriedem and then no timeout?
22:57:44 melwitt I would have thought network_info would remain as it was during a reboot (network_info came from when we originally allocated ports, right?)
22:58:06 mriedem yes network_info should have whatever ports are attached to the instance, regardless of whether or not the vifs are actually plugged
22:58:13 mriedem it's like bdms
22:58:15 mriedem for ports
22:58:25 melwitt good point ... one would think. I'm not that familiar with how the events work. you pass a list of events to the "wait" context manager
22:59:48 melwitt yeah, looks like it would just pass by all the waiting if that list is empty. so the list isn't empty. yet it's not matching up the events
23:03:57 mriedem we should probably log the list of stuff we're registering when we register it
23:04:08 mriedem for debuggabilityamortiziation
23:04:38 melwitt yeah. cause so far this don't make no sense
23:05:07 dansmith you mean log at debug level I assume.. otherwise it's too chatty
23:05:21 mriedem yes debug
23:05:30 melwitt debugaroo
23:05:42 mriedem i can't be chattier than the virt topology shit we log 20 things for
23:05:52 mriedem even though we don't do any stuff with that in the gate
23:05:55 mriedem *it
23:07:42 openstackgerrit Jim Rollenhagen proposed openstack/nova master: Add functional tests for virt driver get_traits() method https://review.openstack.org/538047
23:07:49 jroll mriedem: ^ all yours, that worked out well
23:09:08 openstackgerrit Jim Rollenhagen proposed openstack/nova master: Add functional tests for virt driver get_traits() method https://review.openstack.org/538047
23:09:12 jroll and pep8'd
23:10:30 mriedem damn it
23:10:33 mriedem comment in the last patchset
23:10:43 mriedem seems like those could be one test,
23:10:46 mriedem but idk
23:11:02 melwitt hm, the 'No waiting events found dispatching' message means that the result from _pop_event was no_events_sentinel which has another log message in it 'Unexpected attempt to pop events during shutdown'
23:11:20 mriedem i think that's when the compute service is shutting down
23:11:21 mriedem not the guest
23:11:48 melwitt oh, there's another way to get no_events_sentinel, if no result for self._events.get(instance.uuid)
23:12:15 melwitt so it should be from that
23:15:02 mriedem rloo: TheJulia: jroll: is someone putting out a release? https://review.openstack.org/#/c/532622/
23:15:33 rloo mriedem: TheJulia is in charge of that!
23:15:55 TheJulia we have a reno in the gate that should only take like 10 minutes
23:16:07 mriedem yeah i see
23:16:30 mriedem so, you could start the release patch with a depends-on the reno patch
23:16:36 openstackgerrit Jim Rollenhagen proposed openstack/nova master: Add functional tests for virt driver get_traits() method https://review.openstack.org/538047
23:16:38 mriedem then you just have to mod the git hash once it merges
23:16:57 mriedem i.e. we could start stacking up the dependencies for the release
23:19:41 TheJulia docs and pep8, even less time
23:23:16 openstackgerrit OpenStack Release Bot proposed openstack/python-novaclient master: Update reno for stable/queens https://review.openstack.org/538061
23:29:20 mriedem TheJulia: ship it
23:30:05 mriedem woot
23:30:07 mriedem https://review.openstack.org/538063
23:30:32 TheJulia yup
23:30:42 TheJulia had to just fix an excess space
23:31:06 TheJulia but yup
23:33:19 mriedem jroll: since you've got the series localm
23:33:29 mriedem want to change the depends-on in https://review.openstack.org/#/c/532288/ to https://review.openstack.org/#/c/538066/
23:33:35 mriedem and add the requirements.txt change in nova
23:33:52 mriedem *test-requirements.txt
23:42:38 mriedem the gpu patches failed on the volume backup thing, so i'm going to wait to recheck those
23:44:36 openstackgerrit Matt Riedemann proposed openstack/nova master: Implement get_traits() for the ironic virt driver https://review.openstack.org/532288
23:44:37 openstackgerrit Matt Riedemann proposed openstack/nova master: Add functional tests for virt driver get_traits() method https://review.openstack.org/538047
23:44:37 mriedem jroll: nvm i did it
23:45:49 mriedem dansmith: https://review.openstack.org/#/c/532288/
23:47:16 dansmith mriedem: cool, I was waiting for you to say it was good with that dep
#openstack-nova - 2018-01-26
00:00:22 mriedem q3 tag is up https://review.openstack.org/538072
00:07:35 jroll mriedem: nice one, thanks
00:09:56 SpazZz Will be heading on vacation for about 18 days here soon, any chance that I can get some core love on this sometime soon so it doesn't hit the wayside? https://review.openstack.org/#/c/533168/
00:12:56 tetsuro is efried around?
00:13:12 tetsuro good morning
00:14:47 gmann stephenfin: thanks for adding in ur review list
00:15:45 gmann mriedem: thanks for tempest job, my only left question is can we enable multi-attach in any of the existing job? - https://review.openstack.org/#/c/532689/
00:16:26 gmann bcz new job you are adding as voting
00:18:30 gmann ameeda: sorry i forgot to re-review, i will check today for sure
00:23:19 gmann mriedem: just read your cmt msg for devstack patch about Pike UCA pckg and qemu,l ibvirt version dependency, i think thats why you added separate new job ?
00:29:58 efried tetsuro Hi, briefly around. What's up?
00:31:21 efried mriedem Any point rechecking stuff now, or will it surely fail without the cinder thingy?
00:34:49 openstackgerrit Eric Fried proposed openstack/nova master: New-style _set_inventory_for_provider https://review.openstack.org/537648
00:34:49 openstackgerrit Eric Fried proposed openstack/nova master: SchedulerReportClient.update_from_provider_tree https://review.openstack.org/533821
00:34:50 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Use update_provider_tree from resource tracker https://review.openstack.org/520246
00:34:50 openstackgerrit Eric Fried proposed openstack/nova master: Fix nits in update_provider_tree series https://review.openstack.org/531260
00:34:51 openstackgerrit Eric Fried proposed openstack/nova master: Move refresh time from report client to prov tree https://review.openstack.org/535517
00:36:19 efried tetsuro Bailing out now. If you have the time and inclination, please feel free to recheck stuff all day :P
00:40:00 efried By the way tetsuro-さん, ありがとうございました for all of your excellent and detailed reviews.
00:40:06 tetsuro efried: thank you so much for so kindful and gread explanation!
00:40:30 tetsuro どういたしまして、こちらこそありがとう！笑
00:41:21 efried tetsuro You did point out a bad copy/paste, which I'll fix in the next rev.
00:41:28 efried I'm really leaving now. See y'all tomorrow.
00:41:29 tetsuro ack.
00:41:41 tetsuro have a good night!
01:18:20 gmann mriedem: +1 on nova multiattach job. tempest one need rebase. rechecked job for testing the latest version of tempest test . ll review them accordingly
01:33:59 openstackgerrit melanie witt proposed openstack/nova master: Log the events we timed out waiting for while plugging vifs https://review.openstack.org/538103
02:19:07 openstackgerrit Chen Hanxiao proposed openstack/nova master: test: Use assertEqual() instead of assertXXXEqual() https://review.openstack.org/533493

Earlier   Later