Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-06-14
16:33:32 crazik I think I have some troubles similar to https://bugzilla.redhat.com/show_bug.cgi?id=1403728
16:33:34 openstack bugzilla.redhat.com bug 1403728 in openstack-nova "Unexpected "Instance not resizing, skipping migration" caused by nova evacuation" [Medium,Closed: errata] - Assigned to ratailor
16:33:59 crazik and wonder what are dependencies on this table.
16:34:10 dansmith mriedem: heh, we have no tests for check_can_live_migrate_destination in rpcapi
16:34:15 crazik if no migration is running now
16:39:05 mriedem dansmith: psh who needs tests
16:39:20 mriedem crazik: why do you need migration records for instances that aren't migrating?
16:39:33 mriedem there is an API to see them for one thing, for like auditing and debugging failures
16:39:57 mriedem there are also places in the code that rely on them existing for making decisions, like bringing up an evacuated host
16:40:30 crazik mriedem: I had some issues with migration and need to exclude one reason: old migrations
16:44:03 mriedem crazik: which release are you on?
16:44:17 mriedem the archive stuff was broken a few releases ago
16:44:25 mriedem or, didn't completely cleanup
16:44:40 crazik Pike
16:44:57 crazik well, at migration between Ocata and Pike
16:45:13 crazik few first compute were ok
16:45:27 crazik but next one: few of running - refused to live migrate
16:45:32 crazik (error status in migration)
16:46:12 crazik and ShutOff -also refused migration. done after nova-compute stop and evacuate-host
16:46:54 mriedem if the migration is failing, it's not likely due to old migration records in thedb
16:47:11 mriedem did you look at the nova-compute logs on each host involved in the live migration?
16:47:29 openstackgerrit Dan Smith proposed openstack/nova master: Make check_can_live_migrate_destination use long_rpc_timeout https://review.openstack.org/575493
16:47:31 dansmith mriedem: ^
16:47:32 mriedem could also be scheduler or conductor logs, dependong on where it failed
16:47:38 mriedem dansmith: sweet
16:47:51 dansmith now, I've been up for almost four hours without food, so.. food.
16:48:05 mriedem sustain yourself on a job well done
16:48:07 crazik mriedem: nothing special was there
16:48:23 mriedem crazik: if it failed and went into error state, there must be something in the logs when it failed
16:48:47 crazik hm, I will try to find that.
16:49:00 crazik maybe I skipped something
16:57:35 mriedem another one bites the dust https://blueprints.launchpad.net/nova/+spec/enhanced-kvm-storage-qos
16:57:59 mriedem and https://blueprints.launchpad.net/nova/+spec/xenapi-image-handler-option-improvement
16:59:34 openstackgerrit Eric Fried proposed openstack/nova master: DNM: Leaked nrp-in-alloc-cands behavior https://review.openstack.org/575497
17:16:42 openstackgerrit Elod Illes proposed openstack/nova master: Reject interface attach with QoS aware port https://review.openstack.org/570078
17:16:43 openstackgerrit Elod Illes proposed openstack/nova master: Reject networks with QoS policy https://review.openstack.org/570079
17:23:15 openstackgerrit Matt Riedemann proposed openstack/nova master: Add troubleshooting item about ignored microversions https://review.openstack.org/575506
17:23:16 mriedem melwitt: see if ^ makes sense re jmlowe's microversion issue
17:23:51 dansmith mriedem: dude, hold on,
17:23:57 dansmith she's reviewing my rpc timeout patch right now
17:23:59 dansmith wait your turn
17:24:03 mriedem INTERRUPT
17:24:14 melwitt lol
17:24:17 mriedem dansmith: see if ^ makes sense re jmlowe's microversion issue
17:24:23 dansmith eff.
17:24:26 dansmith that backfired
17:24:29 mriedem ha
17:25:25 dansmith I was too busy being emo in the tc channel to pay attention to that convo
17:26:52 melwitt *group hug*
17:30:29 dansmith mriedem: we could make novaclient look for the server side version headers and warn you if you used a microversion but clearly hit the old api endpoint right?
17:32:44 dansmith or maybe just fail if you asked for a microversion and hit the old api, so you know it's not going to do the thing you asked for
17:35:28 dansmith or maybe that's what you meant by the words around the legacy wrapper thing
17:50:48 mriedem dansmith: more the latter
17:50:55 mriedem doing anything client side is whack a mole
17:58:44 openstackgerrit Chris Dent proposed openstack/nova master: Optional separate database for placement API https://review.openstack.org/362766
17:58:45 openstackgerrit Chris Dent proposed openstack/nova master: Isolate placement database config https://review.openstack.org/541435
17:58:46 openstackgerrit Chris Dent proposed openstack/nova master: Ensure that os-traits sync is attempted only at start of process https://review.openstack.org/553857
17:59:48 openstackgerrit Chris Dent proposed openstack/nova master: Add PLACEMENT_DB_ENABLED=True to the nova-next job https://review.openstack.org/564067
18:12:11 melwitt seems like zuul is stuck
18:20:44 melwitt kashyap: is this blueprint considered complete? it looks like all related changes have merged https://blueprints.launchpad.net/nova/+spec/libvirt-cpu-model-extra-flags
18:30:54 openstackgerrit Matt Riedemann proposed openstack/nova master: Fix nits from trusted certs notification change https://review.openstack.org/575521
18:38:28 efried Sundar: Howdy
18:38:44 efried stephenfin still around by any chance?
18:40:13 Sundar efried: Hi, how are you doing?
18:40:20 efried Living the dream.
18:40:22 Sundar What geo are you in?
18:40:32 efried Central Texas (Austin area)
18:41:04 Sundar "Living the dream" -- great! Reviewing my specs is part of that dream.? :)
18:41:05 efried I was hoping to rope in stephenfin, who knows things about os-vif, but he's UTC.
18:41:33 efried Yeah, I've been sadly behind on that part of the dream, for which I apologize.
18:41:43 efried But trying to catch up now.
18:41:45 Sundar haha -- NP, JK
18:42:03 efried So I only yesterday read up on os-vif, which means two things:
18:42:11 efried 1) It's fresh in my mind
18:42:30 efried 2) It's still really new to me, and I'm sure I don't understand it as deeply as I should.
18:42:33 Sundar I am reviewing os-vif myself. I seem to have missed some important points, like using ovo. Thanks for pointing that out
18:42:40 efried which is why I wanted to bring in a ringer.
18:42:50 Sundar Sure
18:42:53 efried Right, so my understanding is this:
18:43:31 efried We define these base objects - in os-vif, they're subclasses of VIFBase.
18:44:00 efried I'm still not completely clear on what the category of those things is. But I think in the cyborg case, it might be AccBase and then the subclasses might be AccFPGA, AccGPU, AccHPTS, etc.
18:44:07 efried i.e. different broad categories of accelerators.
18:44:26 efried The AccBase has a 'plugin' field.
18:45:25 efried So for example if I want my platform, PowerVM, to provide support for GPUs, maybe I would publish a (separate, third-party) library called acc-powervm or cyborg-powervm or similar.
18:46:01 efried And maybe within that lib I have a class cyborg_powervm.plugin.gpu.PowerVMAccGPUPlugin
18:46:58 efried My package provides some kind of initialization code that registers that plugin and instantiates an instance of os_acc.objects.acc.AccGPU with the `plugin` field set to "cyborg_powervm.plugin.gpu.PowerVMAccGPUPlugin"
18:48:08 edmondsw melwitt mriedem could one of you whack https://blueprints.launchpad.net/nova/+spec/powervm-opts-group ... it's very old and obsolete
18:48:25 Sundar With os-vif and Neutron, most of the functionality is delegated to mechanism/type drivers, agents, etc. So, having a plugin field in each object may make sense. With Cyborg, I'd expect Cyborg agent to do the core work, and rely upon drivers for specifics
18:48:26 efried sorry, the package doesn't instantiate that guy - it just provides a registration hook so that cyborg knows about my plugin. And that registration tells you what kind of accelerator(s) it's good for.
18:49:52 melwitt edmondsw: done, thanks for letting us know
18:49:56 efried Now when cyborg gets a request to attach an accelerator, it knows what kind it is already (let's say GPU) so it knows it needs to instantiate an AccGPU. It looks around for registered plugins and discovers cyborg-powervm is registered; and it looks for plugins registered as being able to handle GPUs and finds cyborg_powervm.plugin.gpu.PowerVMAccGPUPlugin
18:50:14 efried so it sets `plugin` to that value in the AccGPU instance.
18:50:23 Sundar Cyborg already supports a notion of drivers, which cna be per-device or per-vendor, etc. Once the driver has discovered the necessary details, Cyborg has enough info to populate placement, and handle scheduling and attaches. For attaches, it may need to call the driver for specific aspects
18:50:46 Sundar Would/can your library/plugin really be a driver in the Cyborg sense?
18:50:48 efried And from this point on, any time anything platform-specific needs to be done, the plugin gets called upon to do that thing.
18:51:40 efried I don't know whether we're using the same definition of "driver" here, but what I can tell you is that there's no way cyborg knows how to do the plug operation on a Power system.
18:52:12 efried Whoever's doing the plug operation needs to call into Power-specific code.
18:53:08 Sundar You had mentioned that PCI devices in Power are treated differently than in x86. Can you expand on what is different about it?
18:53:21 efried The point of having this AccGPU ovo is that once it's built, you can send it over RPC between cyborg and nova (and maybe others), and as long as those guys have the right stevedore-isms to load up the thing pointed at by the `plugin` field, either side of that equation can invoke the plugin methods when it needs to do something platform-specific.
18:53:24 Sundar May be that will help me understand where the major differences are

Earlier   Later