Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-01-05
15:46:46 hrw melwitt: can you take a look at https://review.openstack.org/#/c/530965/ too?
15:46:46 johnthetubaguy mriedem: if your cache interval is larger than than how long you thing it is dead, it starts to ignore some of your nodes randomly
15:46:54 mriedem ah
15:46:54 hrw melwitt: I am open for any questions
15:46:57 mnaser mriedem: i think i found out how i got that weird state. doing a host-evacuate-live sends a live-migration request for all instances, as nova processes them one by one, the states of all vms in the compute host are in "migrating".. but then its possible that a delete request comes in while the live migration is happening.
15:47:03 sean-k-mooney fried_rice: not upstream, libvirt and qemu dont even have support for vhost-vfio yet. way to early to propose upstrem in openstack
15:47:16 melwitt hrw: I'll add it to my list
15:47:21 hrw melwitt: thanks
15:47:42 hrw melwitt: with those two merged getting openstack on aarch64 is much easier ;D
15:48:10 mnaser in the rpc code, there doesn't seem to be any handling if the VM is in migrating state
15:48:12 sean-k-mooney fried_rice: i would like to have a poc working ptg/early rocky timeframe but i doubt the dependcies will align for it to land even in rocky
15:48:16 mnaser for deletes
15:48:31 hrw melwitt: uefi change is most important as with it in nova we cut 90% of users questions ;d
15:48:58 kashyap mriedem: Hey was AFK
15:49:00 fried_rice sean-k-mooney Okay, gotcha. You going to be around for the sched meeting on Monday? This is exactly the kind of rationale cdent and Radoslav were bringing up in https://review.openstack.org/#/c/526539/ (to pose the question, "Why are we artificially restricting in-tree virt drivers from talking directly to placement?"
15:49:03 kashyap Reading the scroll
15:49:15 fried_rice )
15:49:41 mriedem johnthetubaguy: i don't see a specific config option for how long the cachingscheduler caches it's host state information
15:49:50 mriedem is that somewhere outside the actual CachingScheduler class?
15:49:54 sean-k-mooney fried_rice: am what time is it?( checking meeting page)
15:50:05 johnthetubaguy mriedem: ah, I think its in scheduler manager, sadly
15:50:20 johnthetubaguy just digging
15:50:34 mriedem @periodic_task.periodic_task(spacing=CONF.scheduler.periodic_task_interval,
15:50:34 mriedem run_immediately=True)
15:50:34 mriedem def _run_periodic_tasks(self, context):
15:50:34 mriedem self.driver.run_periodic_tasks(context)
15:50:35 mriedem ?
15:50:50 johnthetubaguy mriedem, yeah https://github.com/openstack/nova/blob/d19e1252d5f2f8e8b54a276003f842ee00641026/nova/scheduler/manager.py#L79
15:50:51 mriedem yup that's it
15:51:08 sean-k-mooney fried_rice: 14:00 UTC i should be able to attend.
15:51:13 openstackgerrit Stephen Finucane proposed openstack/nova master: console: introduce framework for RFB authentication https://review.openstack.org/345397
15:51:14 openstackgerrit Stephen Finucane proposed openstack/nova master: console: introduce the VeNCrypt RFB authentication scheme https://review.openstack.org/345398
15:51:14 openstackgerrit Stephen Finucane proposed openstack/nova master: console: Provide an RFB security proxy implementation https://review.openstack.org/345399
15:51:15 openstackgerrit Stephen Finucane proposed openstack/nova master: doc: Document TLS security setup for noVNC proxy https://review.openstack.org/500544
15:51:16 fried_rice sean-k-mooney Cool.
15:51:23 leakypipes cdent, fried_rice: see my comment on https://review.openstack.org/#/c/526539/
15:51:43 openstackgerrit Matt Riedemann proposed openstack/nova master: [libvirt] Allow multiple volume attachments https://review.openstack.org/267587
15:51:44 openstackgerrit Matt Riedemann proposed openstack/nova master: Pass multiattach flag to reserve_block_device_name https://review.openstack.org/531244
15:51:44 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: [api] Allow multi-attach in compute api https://review.openstack.org/271047
15:51:54 johnthetubaguy mriedem: trying to find the docks I wrote about this, failing :(
15:53:10 stephenfin mriedem: Could you hit this again at some point today? Got an open question on it for you https://review.openstack.org/#/c/526329/
15:53:17 mriedem the config option help for that actually points out the service_down_time option too
15:53:19 cdent leakypipes: thanks reading. I think I tend to use "arbitrary" incorrectly or at least with different connotations that most peole do. I tend to mean it as "something we chose" (as opposed to "a thing we must do")
15:53:22 mriedem so i'm pleasantly surprised
15:53:29 johnthetubaguy mriedem: https://github.com/openstack/nova/blob/d19e1252d5f2f8e8b54a276003f842ee00641026/nova/conf/scheduler.py#L78
15:53:39 johnthetubaguy mriedem: yeah, I knew I put that somewhere!
15:53:43 mriedem johnthetubaguy: yup, was just reading that
15:53:44 mriedem nice
15:53:49 stephenfin oh
15:54:19 leakypipes cdent: sure, understood.
15:54:30 sean-k-mooney fried_rice: im hoping i wont actully need to modify the virt driver as part of the poc beyond the xml generation. on of the advantages of mdevs is you create them using arbitray uuids(neutron port uuid in my case) dynamically so i may be able to skip the resouce tracker entrily as i basically just need a count of howmany can be created per host which the placement inventories will already have.
15:55:35 johnthetubaguy mriedem: there is a type there s/scheduler service as down/compute service as down/
15:55:36 fried_rice leakypipes cdent Good deal. I can see both sides. I don't mind the idea of having intra-nova placement work funneled through this spot for the reasons stated (thread safety, code commonality) as long as we're not doing it with the expectation that e.g. OOT drivers have to conform.
15:55:59 fried_rice sean-k-mooney So where are you going to get that request info from? Just directly from the flavor?
15:56:29 mriedem finucannot: replied
15:56:48 mriedem finucannot: we could backport that to stable/pike upstream, but no further given ocata g-r requires glanceclient>=2.5
15:56:57 mriedem johnthetubaguy: yup i noticed that too
15:57:07 mriedem johnthetubaguy: i'm going to remove the deprecated names for that option, so i'll fix the typo at the same time
15:57:11 leakypipes fried_rice: well, it's Python so of course OOT drivers can always do whatever they want with things ;) but that said, yes, I'd prefer to have the virt drivers just update the ProviderTree object and rely on something else (the reportclient called from the resource tracker most likely) do the work of making placement REST API calls.
15:57:21 finucannot mriedem: Ta
15:57:26 johnthetubaguy mriedem: sweet, thank you
15:57:27 sean-k-mooney fried_rice: of course when i try to do that perhaps ill discovery why mriedem thinks ill what to stabs ye lol. am the requst for the port is encode by the new vnic type + neutron port bining profile info for mdev_type if needed.
15:59:29 fried_rice sean-k-mooney IMO what you're doing here is closer to what should be happening long term. It would be nice if the scheduler and resource tracker could accept requests for resources, allocate them via placement, and pass those allocations through to virt without introspecting or special-casing them at all.
15:59:54 mriedem johnthetubaguy: actually, a bit confused
16:00:18 mriedem johnthetubaguy: it says, "This is because the scheduler driver is responsible for sending a heartbeat and it will only do that as often as this option allows."
16:00:18 fried_rice sean-k-mooney I.e. compute code outside of virt doesn't need to care if a resource is a CPU, GPU, mdev (whatever that is), or peanut.
16:00:29 mriedem johnthetubaguy: i don't see how the compute service down time and the scheduler periodic task interval are linke
16:00:30 mriedem *linked
16:00:53 mriedem sean-k-mooney: you will get stabby because of the constant spelling corrections
16:01:13 mriedem like, "spell knife in the kidneys!"
16:01:58 fried_rice Oh, did I do that? Not consciously.
16:02:05 sean-k-mooney fried_rice: yes i had hopped to work on this as part of the nova neutron negociation stuff in queens and the multiple port binding before we change from upstream first to more poc/internal focoused.
16:03:16 sean-k-mooney mriedem: haha i would have taught i would be the other way round. jay has very kindly help me with this area over the past few years and im very greatful for that.
16:03:35 openstackgerrit Ed Leafe proposed openstack/nova master: Add functional test for resize reschedules https://review.openstack.org/531022
16:03:35 openstackgerrit Ed Leafe proposed openstack/nova master: Change compute RPC to use alternates for resize https://review.openstack.org/526436
16:03:36 openstackgerrit Ed Leafe proposed openstack/nova master: Add regression test for resizing failing during retries https://review.openstack.org/531405
16:03:54 figleaf mriedem: ^^ added regression test and fix
16:04:09 mriedem cool
16:04:12 leakypipes sean-k-mooney: you would have *thought*.
16:04:16 leakypipes sean-k-mooney: you're welcome.
16:04:28 sean-k-mooney leakypipes: :)
16:04:39 leakypipes sean-k-mooney: also... it's *grateful", not greatful. you're welcome, too.
16:04:49 leakypipes sean-k-mooney: :P
16:05:43 kashyap mriedem: Hope you recorded your observation about raw & the shareable thing in the multi-attach review
16:06:00 openstackgerrit melanie witt proposed openstack/nova stable/pike: libvirt: Re-initialise volumes, encryptors, and vifs on hard reboot https://review.openstack.org/531407
16:06:01 kashyap Currently, I'm in split-brains mode, and trying to finishing something before the 'cache is flushed
16:06:43 mriedem kashyap: i did, and it was just something i should have realized wouldn't apply here earlier on
16:06:47 mriedem since imagebackend != volumes
16:08:03 mriedem johnthetubaguy: oh i think i see,
16:08:20 figleaf fried_rice: sorry, missed your question earlier - network issues.
16:08:20 mriedem the ComputeFilter is running every 60 seconds due to the caching scheduler's periodic task interval,
16:08:31 mriedem which means it's potentially using a stale version of the compute service reference,
16:08:33 figleaf fried_rice: mriedem answered what I would have said, so no worries
16:08:36 johnthetubaguy mriedem: sorry, yeah, that's it
16:08:38 mriedem which means it's last_seen_up would be old
16:08:42 johnthetubaguy mriedem: yeah
16:09:22 johnthetubaguy mriedem: hence the note about it not seeing instances being deleted, and being really bad if you run two schedulers
16:09:53 mriedem heh
16:10:03 leakypipes johnthetubaguy: make checks payable to fried_rice.
16:10:04 mriedem if you're wondering why i'm asking https://review.openstack.org/#/c/531020/

Earlier   Later