Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-03-14
20:11:12 efried mriedem Re: "...how the libvirt driver handles connection failures, it will auto-disable the compute service record in the database, and when it reconnects it will auto-enable the compute service record in the DB." - can you give me a pointer to some code to look at (and shamelessly mimic)?
20:18:44 openstackgerrit Juan Antonio Osorio Robles proposed openstack/nova master: Make versioned notifications topics configurable https://review.openstack.org/444947
20:30:43 openstackgerrit Sarafraj Singh proposed openstack/nova-specs master: Spec to add per instance timeout for LM https://review.openstack.org/442809
20:32:35 openstackgerrit Anusha Unnam proposed openstack/nova master: POC: add descriptions to server tags https://review.openstack.org/434842
20:50:45 mriedem dansmith: i failed to mention the van in the yard covered in snow
20:51:01 dansmith mriedem: not that scary
20:51:04 mriedem dansmith: the government center can't do the thing i need, but gene fuckin' turner gets it done
20:51:10 dansmith did it have wheels?
20:51:15 dansmith hah
20:51:16 mriedem gray-faced old ass pug met me at the door
20:51:27 mriedem lots of cat food cans and cat hair but no cat in sight
20:51:33 mriedem the van had wheels
20:51:35 mriedem and rust
20:51:41 mriedem gene mothertruckin' turner
20:51:57 dansmith nice
20:52:01 openstackgerrit Anusha Unnam proposed openstack/nova master: Add descripiton to policies in virtual_interfaces.py https://review.openstack.org/445652
20:52:07 mriedem it was like i saw into jay's future
20:52:18 dansmith haha
20:52:40 mriedem smelled like the mrs was in the kitchen cooking up that cat food
20:52:41 dansmith jay's present is jay's future
20:56:32 mriedem edleafe: this is the callback method in the libvirt driver https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L458
20:56:41 mriedem called from the event listener in nova/virt/libvirt/host.py
20:57:08 mriedem then the magic happens here https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3396
20:57:11 antwash Hey nova team, I have a review for you that's related to the move of policy in code and add more descriptive policies for deployers. https://review.openstack.org/#/c/439070/
20:57:22 antwash ^ feedback is always welcomed :)
20:58:00 edleafe mriedem: thx - will dig into that in a bit
20:58:16 antwash ^ johnthetubaguy -- I think this patch is cleaned up nice now lol
21:02:13 openstackgerrit Sujitha proposed openstack/nova master: Add description to policies in aggregates.py https://review.openstack.org/445230
21:05:18 openstackgerrit Sujitha proposed openstack/nova master: Add description to policies in admin_password.py https://review.openstack.org/443957
21:05:21 openstackgerrit Sujitha proposed openstack/nova master: Add description to policies in admin_password.py https://review.openstack.org/443957
21:07:32 mriedem jroll: i have spoken
21:07:52 jroll joy!
21:08:09 jroll mriedem: nice, thank you much
21:08:17 mriedem i meh'ed all over that thing
21:11:35 jroll same
21:48:02 efried edleafe I suspect mriedem's pointers to _set_host_enabled were for me ;-)
21:48:14 efried If not, I'm interested in talking to you...
21:49:18 efried mriedem Hypothetically, if I wanted to use that thing, could I move it out of libvirt and into someplace common? (Rather than duping it)
21:49:53 efried (Looks like vmware has a [simpler] version of same)
21:52:41 mriedem ha yes they were
21:52:43 mriedem sorry ed
21:53:08 mriedem efried: so vcenter has a way for active session on startup with the backing vcenter?
21:53:11 mriedem *wait
21:53:37 efried mriedem Well, they have a _set_host_enabled that does the same thing as libvirt's. Haven't yet drilled down into how it's used.
21:53:44 mriedem efried: i think we need something like that, otherwise the powervm driver init_host is blocking the nova-compute process from being 'started' for potentially what 10 minutes?
21:53:57 efried mriedem Well, yeah, but...
21:54:09 mriedem it does seem sort of weird to start the nova-compute service but automatically disable it,
21:54:17 mriedem as noted in the review, i'm not sure if the vcenter driver does anything similar
21:54:23 mriedem or how other people feel about this
21:54:25 efried mriedem That service *should* start in under a minute, unless the admin misconfigured the nvl (compute host VM)
21:54:43 efried And I might have other ways of figuring out if that service is trying to start, or dead in the water.
21:55:04 ZZelle Hi everyone
21:55:06 efried (ugly-ass native dbus calls equivalent to systemctl status polling)
21:55:17 ZZelle Who should i ask to review VMware related changes?
21:55:30 ZZelle (in nova of course :))
21:56:03 efried mriedem So... if I could get that polling down to, say, two minutes, with a more reliable dead-in-the-water detection, could I get away with avoiding this subthread compute service disable/enable thing?
21:56:17 mriedem ZZelle: generally you should start with the vmware subteam, which i think right now is browne
21:56:52 ZZelle mriedem, only one person in VMware subteam?
21:56:56 efried mriedem ...which, as you say, seems a bit weird - and fairly involved to validate
21:57:01 browne ZZelle: sure, just put me on the patch to review
21:57:24 ZZelle browne, ok :)
21:57:41 edleafe efried: ah, ok. Thought it was related to another libvirt thing mriedem and I were discussing
21:57:57 efried ZZelle https://etherpad.openstack.org/p/pike-nova-priorities-tracking has a list of subteam members, FYI.
21:58:06 mriedem browne: since you're here, do you know if the vcenter driver has to wait at all to make a connection to the backing vcenter on startup?
21:58:12 efried ZZelle Not sure if your patch would be appropriate for inclusion in that subteam review list.
21:58:44 openstackgerrit Matt Riedemann proposed openstack/nova master: doc: add some documentation around quotas https://review.openstack.org/417563
21:59:34 browne mriedem: i believe there was a bug related to that. where startup of nova-compute was slow because it was connecting to vcenter on startup to retrieve all objects or some effect like that
21:59:40 browne let me do a search for it
21:59:42 efried mriedem https://github.com/openstack/nova/blob/edf51119fa59ff8a3337abb9107a06fa33d3c68f/nova/virt/vmwareapi/host.py#L49-L118
21:59:45 efried and browne ^^
22:00:13 ZZelle efried, i should add my changes in "Ready for subteam review:" part?
22:00:17 efried mriedem browne Looks like on both init and get_host_stats, it checks and updates the service state.
22:00:40 efried ZZelle Not sure if your change set needs to be special to be included there - that's a question for browne et al.
22:00:46 mriedem browne: i remember that too i think, but i might also be thinking of a cache issue with the _sync_instance_power_state periodic task in the compute
22:00:52 ZZelle efried, ok
22:01:08 browne efried: yeah, and i believe someone created a patch to lazy init the state on startup, but wasn't approved
22:01:26 efried browne The above code looks like it kinda does that.
22:01:38 efried lazy and (kinda) polling
22:01:48 browne yeah, so maybe it did get in
22:01:56 efried I like how it cheats by using get_host_stats to kick off the recheck.
22:02:30 efried I can git blame that sucker to determine the provenance...
22:02:46 ZZelle efried, or it's a way to ensure VMware connection during nova-compute-vmware startup?
22:03:36 efried ZZelle Yeah, that's the idea. But it doesn't block; it just marks the compute service as down whenever it detects the external connection is dead - and then re-tests it whenever get_host_stats runs.
22:03:51 efried mriedem and I are trying to figure out if a similar approach is apppropriate for the PowerVM driver.
22:04:05 efried (or if that's more trouble than it's worth)
22:04:49 ZZelle efried, iirc (at least in kilo) the nova-compute crashs if it fails to connect to VMware during service startup
22:05:17 efried ZZelle And that's what the PowerVM external driver does (after trying to connect for 10 minutes)
22:06:18 openstack Launchpad bug 1316928 in OpenStack Compute (nova) "VMware driver does not change compute node state to unavailable after disconnecting from vCenter" [Medium,Fix released] - Assigned to Giridhar Jayavelu (gjayavelu)
22:06:18 efried ZZelle mriedem browne Looks like it happened here https://review.openstack.org/#/c/290315/ as a result of https://launchpad.net/bugs/1316928
22:08:57 browne efried: i was thinking of a different bug, but this one looks familiar too
22:11:30 efried So mriedem, I'm thinking this is a pretty involved bit of stuff to try to drop into the initial patch. Whaddaya say I knock that timeout down to like two minutes, and develop the state changer thingy in a subsequent change set?
22:17:07 mriedem efried: sure, and leave a TODO in the initial patch or whatever
22:17:15 efried mriedem Cool beans, thanks.
22:17:23 mriedem bmace: let's chat here,
22:17:40 mriedem bmace: there is a weekly nova/cinder meeting http://eavesdrop.openstack.org/#Cinder,_Nova
22:17:55 mriedem bmace: for anyone interested in the multiattach stuff, that's a good meeting to be in for latest status and what needs attention or help
22:18:19 bmace mriedem: thanks much
22:18:46 mriedem bmace: we're tracking review priorities per subteam in here https://etherpad.openstack.org/p/pike-nova-priorities-tracking
22:19:06 mriedem cinder stuff is at L75
22:19:21 mriedem i'm trying to keep as much of it contained to per-week increments as possible,

Earlier   Later