Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-03-16
16:23:54 dansmith jroll: sure, but if you're going to include all that work in here, you probably need to add some wordy description of the allocation bits into the proposed change as well, because that's probably why I assumed this was only the flavor/scheduling work
16:24:21 jroll gotcha
16:24:53 jroll dansmith: I have no clue how that thing works, I'll add as a dep for now until I chat with jay
16:26:09 dansmith okay if he wants it all in this spec I'm sure he can just add it himself
16:27:39 jroll ya
16:28:34 mriedem lyarwood: so i was thinking, we should probably decouple the attachment_id and the rest of that series from the bdm.uuid patch, since that's getting overly complicated at this point
16:28:55 mriedem lyarwood: and it's been about 2 weeks, can we just decouple that from the series?
16:29:26 mriedem melwitt: adding the RetryFilter in makes it blow up for some reason
16:29:47 mriedem melwitt: you could pull it down and add that to see what i mean, but it seems to think that there is only one host,
16:30:00 mriedem i'm not sure if that's because both compute services have the same fake-mini node?
16:30:46 melwitt mriedem: yeah, I suspect it's trying to same host again. it still tests the bug since it is retrying, it's just probably not landing on the second host
16:31:08 lyarwood mriedem: yup happy to, I'll do that now before the call
16:31:10 mriedem melwitt: well, it dumps a message like "host1 in list, hosts already tried: host1"
16:31:19 lyarwood s/call/meeting/g
16:31:20 mriedem and then kicks it out,
16:31:30 melwitt mriedem: oh? I pulled it down a few minutes ago and am playing with it, just out of curiosity
16:31:30 mriedem i'm not sure why it doesn't see host2
16:31:47 mriedem my guess is the node values are the same, but i don't see why that should matter
16:31:59 openstackgerrit Jim Rollenhagen proposed openstack/nova-specs master: Add spec for custom resource classes in flavors https://review.openstack.org/446570
16:32:00 mriedem lyarwood: thanks
16:32:03 jroll dansmith: done, thanks for the help
16:43:35 jaypipes mhenkel: where is the os-vif plugin for vrouter?
16:45:00 jaypipes mhenkel: nm, found it, sorry.
16:45:18 mhenkel jaypipes: cool
16:45:36 openstackgerrit Octave Orgeron proposed openstack/nova-specs master: Enables MySQL Cluster Support for Nova https://review.openstack.org/446626
16:55:38 openstackgerrit Octave Orgeron proposed openstack/nova-specs master: Enables MySQL Cluster Support for Nova https://review.openstack.org/446631
16:56:40 openstackgerrit Stephen Finucane proposed openstack/python-novaclient master: Use Sphinx 1.5 warning-is-error https://review.openstack.org/446632
16:58:41 jaypipes mhenkel: still -1 from me, but I understand now where the os-vif stuff is. :)
16:58:50 jaypipes mhenkel: pls see my review comments for details.
16:59:14 mhenkel jaypipes: ok, will check, thanks!
16:59:16 openstackgerrit Lee Yarwood proposed openstack/nova master: objects: Add attachment_id to BlockDeviceMapping https://review.openstack.org/437665
16:59:17 openstackgerrit Lee Yarwood proposed openstack/nova master: db: Add attachment_id to block_device_mapping https://review.openstack.org/437597
16:59:22 jaypipes mhenkel: np! :)
17:00:04 jaypipes mhenkel: if you add those unit tests and make the commit message changes, I'll re-review quickly and with sean-k-mooney's +1 pass it over to sfinucan or mriedem to +2.
17:00:29 mhenkel jaypipes: will do!
17:01:58 jaypipes mriedem: https://review.openstack.org/#/c/416669/ looks sensible and has had several cdent reviews around API correctness.
17:02:04 openstackgerrit John Garbutt proposed openstack/nova-specs master: Add spec to use cinder's new attachment API https://review.openstack.org/373203
17:02:20 sean-k-mooney jaypipes: i proably should have been more stringent in my own review but the main think i was reviewing for was if the os-vif delegation looked correct.
17:02:35 jaypipes sean-k-mooney: it's cool duder :)
17:02:37 mdbooth mriedem lyarwood: I've been looking at that detach gate failure, discussing with danpb downstream.
17:03:04 mdbooth I currently suspect it's a libvirt issue.
17:03:40 mdbooth If you look at the libvirt logs, we see the device_del, then the job disappears into a black hole, then Nova reports a disconnect.
17:03:47 sean-k-mooney jaypipes: that said alot of that code that deligate to os-vif in nova is not unittested correctly so this is something i guess we should improve going forward
17:04:10 jaypipes sean-k-mooney: ack
17:04:26 mriedem jaypipes: ok, after meetings and other things, like this big ass bowl of chili in front of me
17:05:25 mdbooth lyarwood mriedem: All of the above occurs within the context of, from Nova's pov, self._domain.detachDeviceFlags(device_xml, flags=flags)
17:05:37 mdbooth So at the moment I don't see how it can be a Nova issue
17:06:14 sean-k-mooney mhenkel: the unitest in https://review.openstack.org/#/c/334048/ may be of use to you as a reference.
17:07:55 mdbooth mriedem: How long do those log files live, btw? If I link to them in bz will they still be there in a month or 2?
17:09:20 mriedem mdbooth: i'm not sure about that, it used to be 6 months,
17:09:35 mriedem fungi: jeblair: ^ how long does infra keep ci job logs around?
17:09:53 fungi mriedem: now it's down to about 45 days
17:10:08 fungi mdbooth: ^
17:10:15 jaypipes mriedem: :)
17:10:16 mdbooth fungi: Thanks
17:10:36 sean-k-mooney jaypipes: actully while you are around if you get a chance to look at https://review.openstack.org/#/c/441590/ let me know if you are ok with this direction. i belive that with this change the host_info class has all the info we need to pass to neutron for the port binding negciation.
17:10:52 fungi mriedem: mdbooth: that's keeping us barely afloat with the present job log volume http://cacti.openstack.org/cacti/graph.php?action=view&local_graph_id=717&rra_id=all
17:10:57 jeblair (it's directly tied to log volume; we've had to reduce it due to a significant increase in the amount of data jobs are storing in logs)
17:11:05 sean-k-mooney jaypipes: that said i dont want to merge this until i have some prototype code to consume it
17:11:09 mriedem mdbooth: fwiw i didn't notice this type of error when we were using trusty
17:11:12 mriedem which was libvirt 1.2.2
17:11:55 mdbooth fungi: NP. I originally assumed it would be days, tbh.
17:12:01 mdbooth 45 days is excellent.
17:12:30 sean-k-mooney sfinucan: https://review.openstack.org/#/c/441590/ is proably of interest to you too. again im going to keep the workflow -1 until i have an end to end poc working to ensure it does capture all required info
17:14:12 fungi mdbooth: we wish it could be more, but we generate _lots_ of logs these days
17:19:52 openstackgerrit Christopher Brown proposed openstack/nova master: Add lan9118 as valid nic for hw_vif_model property for qemu https://review.openstack.org/393489
17:22:03 jaypipes sean-k-mooney: reviewed.
17:25:19 mriedem fungi: totally fine
17:25:24 mriedem was just wondering the timeline
17:25:47 openstackgerrit Octave Orgeron proposed openstack/nova master: Enables MySQL Cluster Support for Nova https://review.openstack.org/446643
17:27:14 sean-k-mooney jaypipes: thanks that also needs units so ill adress your comments when i add them
17:27:35 openstackgerrit Octave Orgeron proposed openstack/nova master: Enables MySQL Cluster Support for Nova https://review.openstack.org/446643
17:33:25 mdbooth lyarwood mriedem: libvirtd crashed
17:34:23 mriedem mdbooth: gah
17:34:33 mriedem remember how i said i'd always chalked this failure up to libvirt crashing :)
17:34:55 mdbooth mriedem: So, I think we're going to request some changes in the CI environment
17:35:10 mdbooth e.g. It would be nice if libvirt didn't restart
17:35:20 mdbooth Also, if we collected core dumps
17:35:28 mdbooth And there are some additional logs which would be useful
17:35:34 openstackgerrit Dan Smith proposed openstack/nova master: Teach HostAPI about cells https://review.openstack.org/442162
17:35:35 openstackgerrit Dan Smith proposed openstack/nova master: Make scheduler target cells to get compute node instance info https://review.openstack.org/439891
17:35:39 mdbooth mriedem: What's the best way to request ^^^ ?
17:35:39 mriedem i.e. http://status.openstack.org/elastic-recheck/#1643911 http://status.openstack.org/elastic-recheck/#1646779 http://status.openstack.org/elastic-recheck/#1638982
17:36:02 mriedem mdbooth: i think the libvirtd logging stuff is handled in devstack
17:37:56 mriedem mdbooth: https://github.com/openstack-dev/devstack/blob/master/lib/nova_plugins/functions-libvirt#L104
17:38:15 mdbooth http://logs.openstack.org/75/446175/1/gate/gate-tempest-dsvm-neutron-full-ubuntu-xenial/479c7bf/logs/syslog.txt.gz#_Mar_16_01_01_32
17:38:16 mdbooth btw
17:38:47 openstackgerrit John Garbutt proposed openstack/nova master: compute: Only destroy BDMs after successful detach call https://review.openstack.org/440693
17:38:51 cfriesen is anyone aware of a way to have libvirt/qemu live migration maintain the "sparseness" of a qcow2 disk file? After live migration I'm seeing the physical size match the virtual size even though most of that space is not actually used.
17:42:23 mdbooth cfriesen: Is this an image-backed qcow2?
17:42:30 mdbooth Because if so, it should already do that.
17:44:25 mriedem mdbooth: ok so we can just duplicate this bug i guess,
17:44:29 mriedem and i wasted everyone's time
17:44:35 mriedem but thanks for digging into it,
17:44:44 mriedem if we can get more details on the actual crashes, that'd be great,
17:44:45 openstackgerrit sean mooney proposed openstack/nova master: remove flake8-import-order for test requirements https://review.openstack.org/445622
17:44:54 mdbooth mriedem: We should raise a bug against Ubuntu's libvirt package
17:44:57 mriedem becaues it's a pretty high probability gate failure since we moved to xenial
17:45:08 mriedem i wouldn't really know what to say in the bug,
17:45:13 mriedem except "it crashes :P "

Earlier   Later