| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-31 | |||
| 11:09:15 | andymccr | johnthetubaguy: hmm yeah, so that pretty much lines up - i tested with --conf-file but let me test that again | |
| 11:09:23 | sfinucan | artom, mdbooth: ^^^ | |
| 11:09:46 | johnthetubaguy | andymccr: this is our nova.conf in the gate: http://logs.openstack.org/63/426163/2/check/gate-tempest-dsvm-neutron-full-ubuntu-trusty/a5ece9b/logs/etc/nova/nova.conf.txt.gz | |
| 11:10:39 | openstack | Launchpad bug 1658877 in OpenStack Compute (nova) "live migration failed with XenServer as hypervisor" [High,In progress] - Assigned to Bob Ball (bob-ball) | |
| 11:10:39 | BobBall | mriedem: See my updates to https://bugs.launchpad.net/nova/+bug/1658877 . I'd very much hope we can still aim for a fix in Ocata as changes to the object are needed and I understand these are particularly difficult to backport? | |
| 11:11:09 | mdbooth | sfinucan: You want to add a test for ignoring EPERM? ;) | |
| 11:11:19 | johnthetubaguy | andymccr: did you configure api_database with the placement database, and use the correct conf file when calling api_sync? | |
| 11:11:30 | sfinucan | mdbooth: Damn - you're quick. Had just started fixing that :) | |
| 11:12:05 | andymccr | johnthetubaguy: yes on the config file, but we have a the placement_database section defined. | |
| 11:12:26 | johnthetubaguy | andymccr: something tells me we screwed up, and nova-manage doesn't read that | |
| 11:13:01 | andymccr | johnthetubaguy: interesting, let me test and remove that section | |
| 11:13:06 | andymccr | i assume it'll just default to the api_database then? | |
| 11:13:22 | johnthetubaguy | andymccr: yeah, we will want to not do that, but it would be good to see that fix your issues | |
| 11:14:04 | andymccr | johnthetubaguy: although - i know the puppet team has dedicated configuration for nova_placement database. | |
| 11:14:28 | andymccr | but lets test | |
| 11:14:35 | andymccr | im somewhat sure its db related | |
| 11:14:37 | johnthetubaguy | andymccr: yeah, we should do a separate database, but it would be good to see if that fixes things | |
| 11:16:57 | openstackgerrit | Takashi NATSUME proposed openstack/nova: Avoid forcing translation on logging calls https://review.openstack.org/413876 | |
| 11:18:26 | johnthetubaguy | bauzas: have you noticed how you can't have a separate placement database using the placement_database settings? seems in the gate we test with a single shared API database | |
| 11:24:18 | openstackgerrit | Stephen Finucane proposed openstack/nova: Ignore IOError when creating 'console.log' https://review.openstack.org/392643 | |
| 11:31:34 | openstackgerrit | Andy McCrae proposed openstack/nova: Allow placement endpoint interface to be set https://review.openstack.org/426163 | |
| 11:38:54 | rpodolyaka | johnthetubaguy: yeah, a separate placement db must only be possible with https://review.openstack.org/#/c/362766/ | |
| 11:40:21 | rpodolyaka | I wonder if we left placement_database conf group intentionally in this revert - https://review.openstack.org/#/c/360638/ or by mistake | |
| 11:42:26 | johnthetubaguy | rpodolyaka: ah, right, I couldn't see that code when I went digging | |
| 11:43:45 | johnthetubaguy | andymccr: so I think we could make the placement DB separate by having a placement.conf that is used for the nova-manage call and the placement API, where the api-database is configured as the placement database | |
| 11:44:04 | johnthetubaguy | andymccr: seems we are missing some bits on our side for that to work... more sensibly | |
| 11:45:10 | openstackgerrit | Abhishek Kekane proposed openstack/nova: Use is_valid_cidr and is_valid_ipv6_cidr from oslo_utils https://review.openstack.org/320375 | |
| 11:46:51 | andymccr | johnthetubaguy: ahh ok - weird. i wonder how the puppet stuff works then - im running a few tests now wrt not using a set placement_db | |
| 11:47:09 | johnthetubaguy | andymccr: +1 | |
| 11:47:20 | andymccr | i'll try speak to EmilienM a bit later - he put the pr in for that :) | |
| 11:48:18 | sfinucan | lyarwood: Could you say if this is suitable bugfix to backport to Mitaka? https://review.openstack.org/427119 | |
| 11:48:31 | openstack | Launchpad bug 1578155 in OpenStack Compute (nova) newton "duplicate for #1657791 'hw:cpu_thread_policy=prefer' misbehaviour" [Medium,Fix committed] - Assigned to Stephen Finucane (stephenfinucane) | |
| 11:48:31 | sfinucan | Per this bug report https://bugs.launchpad.net/nova/+bug/1657791 | |
| 11:57:58 | tdurakov | johnthetubaguy: could you please take a look https://review.openstack.org/#/c/244489/58 especially interested in default rpc method params, see my comments on ps-58, is it really required? I would like to clarify that moment for myself | |
| 12:00:58 | johnthetubaguy | andymccr: on a related note, not sure if you have the cell0 database setup yet, there are details in here: http://docs.openstack.org/developer/nova/cells.html#setup-of-cells-v2 | |
| 12:01:33 | andymccr | johnthetubaguy: i was looking at that - is that a requirement? so you have to have a cell0 db (and a cell1 db) | |
| 12:01:50 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Add extra-specs-image-meta-conflicts spec https://review.openstack.org/382382 | |
| 12:02:17 | johnthetubaguy | andymccr: yes, its all required now, I believe | |
| 12:02:23 | andymccr | johnthetubaguy: i wonder if that is the issue then. | |
| 12:02:24 | andymccr | hmm | |
| 12:02:38 | andymccr | ok cool that gives me some more things to try at the lest | |
| 12:02:42 | andymccr | *least | |
| 12:03:22 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Reserve NUMA nodes with PCI devices attached https://review.openstack.org/364468 | |
| 12:03:34 | johnthetubaguy | andymccr: you need the API db to know which cell a host is in, else we don't know which cell to send the build request, etc | |
| 12:04:25 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Add 'attributed-action-notifications' spec https://review.openstack.org/388573 | |
| 12:04:58 | lyarwood | sfinucan: hey sorry was on a call, no I don't think it is http://docs.openstack.org/project-team-guide/stable-branches.html#support-phases | |
| 12:06:13 | sfinucan | lyarwood: I thought as much. I was hoping the clause might apply though :) " It’s nevertheless allowed to backport fixes for other bugs if their safety can be easily proved. " | |
| 12:07:44 | lyarwood | sfinucan: yeah we do make exceptions for simple fixes, I'm just going by the bug subject and Importance posted in the channel | |
| 12:10:00 | openstack | Launchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed] | |
| 12:10:00 | kashyap | Hmm, this has become a bit of blocker it seems -- https://bugs.launchpad.net/bugs/1643911 | |
| 12:10:57 | kashyap | And the upstream libvirt folks go: "the stack trace will be totally useless for malloc memory corruption" | |
| 12:12:19 | openstack | Launchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed] | |
| 12:12:19 | kashyap | Because: https://bugs.launchpad.net/nova/+bug/1643911/comments/1 | |
| 12:12:37 | kashyap | If you run libvirtd under valgrind in the Gate, then it might hide the problem | |
| 12:13:02 | kashyap | At least libvirt upstream admits: "I mean, we [libvirt] are clearly overwriting the memory somewhere; but stack trace is not helpful in those cases" | |
| 12:32:24 | openstackgerrit | Ghanshyam Mann proposed openstack/nova: Add release note and docs for filter/sort whitelist https://review.openstack.org/421760 | |
| 12:37:54 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova: Optimize time during PCI alias parsing process https://review.openstack.org/427145 | |
| 13:07:26 | openstackgerrit | Emilien Macchi proposed openstack/nova: DNM - tripleo CI test https://review.openstack.org/426949 | |
| 13:13:19 | cdent | johnthetubaguy, andymccr: Did you guys work out what was going on a while back with the placement db or are things still weird? | |
| 13:13:48 | cdent | andymccr: I'm pretty sure the original 409 that is happening is because the generation on the resource provider is run, but it corrects itself so the next writes out to work | |
| 13:13:53 | cdent | s/out/ought/ | |
| 13:14:06 | cdent | (they do for me when I've been testing things in devstack) | |
| 13:14:24 | cdent | so your theory that it is a database problem is likely | |
| 13:17:19 | johnthetubaguy | cdent: I think the DB wasn't configured correctly | |
| 13:17:34 | johnthetubaguy | cdent: but I could be wrong, not seen andymccr latest results | |
| 13:18:02 | johnthetubaguy | cdent: I forgot we didn't merge the stuff to support the separate placement_database | |
| 13:18:59 | andymccr | johnthetubaguy: cdent - thanks for the info :) I'm testing some more things with database changes/perms etc - will update when i (hopefully) have something working! | |
| 13:19:03 | cdent | yeah, it totally stalled out. I have a working change that I keep up to date for it, but people stated unspecific reservations about it, without specifics there was no way to move on... | |
| 13:19:54 | johnthetubaguy | cdent: jaypipes: I still had worries about the allocations in that ironic patch, I added comments/suggestions | |
| 13:20:04 | cdent | johnthetubaguy: that's next | |
| 13:20:26 | johnthetubaguy | cool, thanks | |
| 13:20:28 | cdent | I skimmed your comments earlier today and my immediate thought was "moar tests" | |
| 13:21:12 | markus_z | mriedem: Would you please remove/re-evaluate your -1 on https://review.openstack.org/#/c/406897/ ? | |
| 13:22:40 | openstackgerrit | Andy McCrae proposed openstack/nova: Allow placement endpoint interface to be set https://review.openstack.org/426163 | |
| 13:23:06 | openstackgerrit | Sylvain Bauza proposed openstack/nova: Add a status check for enabled filters https://review.openstack.org/427200 | |
| 13:24:47 | openstack | Launchpad bug 1660317 in ironic (Ubuntu) "NotImplementedError for detach_interface in nova-compute during instance deletion" [Undecided,New] | |
| 13:24:47 | sfinucan | Kevin_Zheng: Could you take a look at https://bugs.launchpad.net/nova/+bug/1660317 ? fwict, the error message that they're receiving should be mitigated by a patch of yours | |
| 13:25:01 | sfinucan | Namely, this patch https://github.com/openstack/nova/commit/4ad414f3b1216393301ef268a64e61ca1a3d5be9 | |
| 13:25:15 | sfinucan | ...though I could be reading that incorrectly :) | |
| 13:26:02 | mriedem | sfinucan: have you seen https://review.openstack.org/#/c/403925/ ? | |
| 13:26:18 | mriedem | markus_z: removed the -1 | |
| 13:26:21 | mriedem | until i can dig deeper | |
| 13:26:44 | markus_z | mriedem: Thanks | |
| 13:27:26 | sfinucan | mriedem: I had not. I can take a look now | |
| 13:28:18 | Kevin_Zheng | sfinucan: sorry, I'm on vacation and didn't got my computer with me:( | |
| 13:28:30 | mriedem | sfinucan: just wondering if it's related to what you're asking about | |
| 13:28:34 | bauzas | mriedem: FYI https://review.openstack.org/427200 | |
| 13:28:52 | sfinucan | Kevin_Zheng: No problem at all - it's not that important. Enjoy your vacation :) | |
| 13:29:16 | sfinucan | mriedem: I'm looking but I don't think so | |
| 13:29:28 | Kevin_Zheng | Thanks | |
| 13:32:29 | openstackgerrit | OpenStack Proposal Bot proposed openstack/os-vif: Updated from global requirements https://review.openstack.org/395347 | |
| 13:37:20 | mriedem | toabctl: +2 on https://review.openstack.org/#/c/402318/ with some comments - the libvirt driver ignores the requested device name that tempest is sending in anyway | |
| 13:37:46 | mriedem | so if the libvirt driver was always defaulting to vda i'm not sure how things worked with the xenproject ci | |
| 13:38:35 | mriedem | doesn't look like the libvirt+xen ci overrides that in tempest.conf either http://logs.openstack.xenproject.org/18/402318/1/check/dsvm-tempest-xen/6b9829a/logs/tempest_conf.txt.gz | |
| 13:39:44 | mriedem | anthonyper: ^ | |
| 13:39:47 | toabctl | mriedem, is libvirt+xen tested? | |
| 13:39:54 | mriedem | yes | |
| 13:39:57 | mriedem | that's the xenproject ci | |
| 13:41:00 | mriedem | 2016-11-25 08:10:13.591 13563 WARNING nova.virt.libvirt.driver [req-98364dbb-0b80-49f3-9d59-ac476fdd3855 tempest-AttachVolumeShelveTestJSON-134397458 tempest-AttachVolumeShelveTestJSON-134397458] [instance: b0cf5c0f-ae34-4ca1-b441-08a3f4a85b71] Ignoring supplied device name: /dev/vdb | |
| 13:41:00 | mriedem | e.g. | |