| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-01-31 | |||
| 11:48:31 | sfinucan | Per this bug report https://bugs.launchpad.net/nova/+bug/1657791 | |
| 11:48:31 | openstack | Launchpad bug 1578155 in OpenStack Compute (nova) newton "duplicate for #1657791 'hw:cpu_thread_policy=prefer' misbehaviour" [Medium,Fix committed] - Assigned to Stephen Finucane (stephenfinucane) | |
| 11:57:58 | tdurakov | johnthetubaguy: could you please take a look https://review.openstack.org/#/c/244489/58 especially interested in default rpc method params, see my comments on ps-58, is it really required? I would like to clarify that moment for myself | |
| 12:00:58 | johnthetubaguy | andymccr: on a related note, not sure if you have the cell0 database setup yet, there are details in here: http://docs.openstack.org/developer/nova/cells.html#setup-of-cells-v2 | |
| 12:01:33 | andymccr | johnthetubaguy: i was looking at that - is that a requirement? so you have to have a cell0 db (and a cell1 db) | |
| 12:01:50 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Add extra-specs-image-meta-conflicts spec https://review.openstack.org/382382 | |
| 12:02:17 | johnthetubaguy | andymccr: yes, its all required now, I believe | |
| 12:02:23 | andymccr | johnthetubaguy: i wonder if that is the issue then. | |
| 12:02:24 | andymccr | hmm | |
| 12:02:38 | andymccr | ok cool that gives me some more things to try at the lest | |
| 12:02:42 | andymccr | *least | |
| 12:03:22 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Reserve NUMA nodes with PCI devices attached https://review.openstack.org/364468 | |
| 12:03:34 | johnthetubaguy | andymccr: you need the API db to know which cell a host is in, else we don't know which cell to send the build request, etc | |
| 12:04:25 | openstackgerrit | Stephen Finucane proposed openstack/nova-specs: Add 'attributed-action-notifications' spec https://review.openstack.org/388573 | |
| 12:04:58 | lyarwood | sfinucan: hey sorry was on a call, no I don't think it is http://docs.openstack.org/project-team-guide/stable-branches.html#support-phases | |
| 12:06:13 | sfinucan | lyarwood: I thought as much. I was hoping the clause might apply though :) " It’s nevertheless allowed to backport fixes for other bugs if their safety can be easily proved. " | |
| 12:07:44 | lyarwood | sfinucan: yeah we do make exceptions for simple fixes, I'm just going by the bug subject and Importance posted in the channel | |
| 12:10:00 | kashyap | Hmm, this has become a bit of blocker it seems -- https://bugs.launchpad.net/bugs/1643911 | |
| 12:10:00 | openstack | Launchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed] | |
| 12:10:57 | kashyap | And the upstream libvirt folks go: "the stack trace will be totally useless for malloc memory corruption" | |
| 12:12:19 | kashyap | Because: https://bugs.launchpad.net/nova/+bug/1643911/comments/1 | |
| 12:12:19 | openstack | Launchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed] | |
| 12:12:37 | kashyap | If you run libvirtd under valgrind in the Gate, then it might hide the problem | |
| 12:13:02 | kashyap | At least libvirt upstream admits: "I mean, we [libvirt] are clearly overwriting the memory somewhere; but stack trace is not helpful in those cases" | |
| 12:32:24 | openstackgerrit | Ghanshyam Mann proposed openstack/nova: Add release note and docs for filter/sort whitelist https://review.openstack.org/421760 | |
| 12:37:54 | openstackgerrit | Rodolfo Alonso Hernandez proposed openstack/nova: Optimize time during PCI alias parsing process https://review.openstack.org/427145 | |
| 13:07:26 | openstackgerrit | Emilien Macchi proposed openstack/nova: DNM - tripleo CI test https://review.openstack.org/426949 | |
| 13:13:19 | cdent | johnthetubaguy, andymccr: Did you guys work out what was going on a while back with the placement db or are things still weird? | |
| 13:13:48 | cdent | andymccr: I'm pretty sure the original 409 that is happening is because the generation on the resource provider is run, but it corrects itself so the next writes out to work | |
| 13:13:53 | cdent | s/out/ought/ | |
| 13:14:06 | cdent | (they do for me when I've been testing things in devstack) | |
| 13:14:24 | cdent | so your theory that it is a database problem is likely | |
| 13:17:19 | johnthetubaguy | cdent: I think the DB wasn't configured correctly | |
| 13:17:34 | johnthetubaguy | cdent: but I could be wrong, not seen andymccr latest results | |
| 13:18:02 | johnthetubaguy | cdent: I forgot we didn't merge the stuff to support the separate placement_database | |
| 13:18:59 | andymccr | johnthetubaguy: cdent - thanks for the info :) I'm testing some more things with database changes/perms etc - will update when i (hopefully) have something working! | |
| 13:19:03 | cdent | yeah, it totally stalled out. I have a working change that I keep up to date for it, but people stated unspecific reservations about it, without specifics there was no way to move on... | |
| 13:19:54 | johnthetubaguy | cdent: jaypipes: I still had worries about the allocations in that ironic patch, I added comments/suggestions | |
| 13:20:04 | cdent | johnthetubaguy: that's next | |
| 13:20:26 | johnthetubaguy | cool, thanks | |
| 13:20:28 | cdent | I skimmed your comments earlier today and my immediate thought was "moar tests" | |
| 13:21:12 | markus_z | mriedem: Would you please remove/re-evaluate your -1 on https://review.openstack.org/#/c/406897/ ? | |
| 13:22:40 | openstackgerrit | Andy McCrae proposed openstack/nova: Allow placement endpoint interface to be set https://review.openstack.org/426163 | |
| 13:23:06 | openstackgerrit | Sylvain Bauza proposed openstack/nova: Add a status check for enabled filters https://review.openstack.org/427200 | |
| 13:24:47 | sfinucan | Kevin_Zheng: Could you take a look at https://bugs.launchpad.net/nova/+bug/1660317 ? fwict, the error message that they're receiving should be mitigated by a patch of yours | |
| 13:24:47 | openstack | Launchpad bug 1660317 in ironic (Ubuntu) "NotImplementedError for detach_interface in nova-compute during instance deletion" [Undecided,New] | |
| 13:25:01 | sfinucan | Namely, this patch https://github.com/openstack/nova/commit/4ad414f3b1216393301ef268a64e61ca1a3d5be9 | |
| 13:25:15 | sfinucan | ...though I could be reading that incorrectly :) | |
| 13:26:02 | mriedem | sfinucan: have you seen https://review.openstack.org/#/c/403925/ ? | |
| 13:26:18 | mriedem | markus_z: removed the -1 | |
| 13:26:21 | mriedem | until i can dig deeper | |
| 13:26:44 | markus_z | mriedem: Thanks | |
| 13:27:26 | sfinucan | mriedem: I had not. I can take a look now | |
| 13:28:18 | Kevin_Zheng | sfinucan: sorry, I'm on vacation and didn't got my computer with me:( | |
| 13:28:30 | mriedem | sfinucan: just wondering if it's related to what you're asking about | |
| 13:28:34 | bauzas | mriedem: FYI https://review.openstack.org/427200 | |
| 13:28:52 | sfinucan | Kevin_Zheng: No problem at all - it's not that important. Enjoy your vacation :) | |
| 13:29:16 | sfinucan | mriedem: I'm looking but I don't think so | |
| 13:29:28 | Kevin_Zheng | Thanks | |
| 13:32:29 | openstackgerrit | OpenStack Proposal Bot proposed openstack/os-vif: Updated from global requirements https://review.openstack.org/395347 | |
| 13:37:20 | mriedem | toabctl: +2 on https://review.openstack.org/#/c/402318/ with some comments - the libvirt driver ignores the requested device name that tempest is sending in anyway | |
| 13:37:46 | mriedem | so if the libvirt driver was always defaulting to vda i'm not sure how things worked with the xenproject ci | |
| 13:38:35 | mriedem | doesn't look like the libvirt+xen ci overrides that in tempest.conf either http://logs.openstack.xenproject.org/18/402318/1/check/dsvm-tempest-xen/6b9829a/logs/tempest_conf.txt.gz | |
| 13:39:44 | mriedem | anthonyper: ^ | |
| 13:39:47 | toabctl | mriedem, is libvirt+xen tested? | |
| 13:39:54 | mriedem | yes | |
| 13:39:57 | mriedem | that's the xenproject ci | |
| 13:41:00 | mriedem | e.g. | |
| 13:41:00 | mriedem | 2016-11-25 08:10:13.591 13563 WARNING nova.virt.libvirt.driver [req-98364dbb-0b80-49f3-9d59-ac476fdd3855 tempest-AttachVolumeShelveTestJSON-134397458 tempest-AttachVolumeShelveTestJSON-134397458] [instance: b0cf5c0f-ae34-4ca1-b441-08a3f4a85b71] Ignoring supplied device name: /dev/vdb | |
| 13:45:48 | mriedem | toabctl: dropped to a +1 until someone that knows about the xenproject ci can explain why this wasn't a failure before | |
| 13:46:56 | mriedem | maybe it's just because that ci wasn't running the test in question, test_volume_boot_pattern | |
| 13:47:37 | mriedem | anthonyper: is the xenproject ci whitelist/blacklist available somewhere? i'm wondering if i could enable test_volume_boot_pattern against https://review.openstack.org/#/c/402318/ | |
| 13:47:43 | BobBall | DEVSTACK_GATE_TEMPEST_REGEX='(?!.*\[.*\bslow\b.*\]|.*TestVolumeBootPattern\.test_(volume_boot_pattern|create <-- confirmed | |
| 13:48:08 | mriedem | where is that defined though? | |
| 13:48:36 | BobBall | Defined here http://xenbits.xen.org/gitweb/?p=openstack/ci-loop-config.git;a=blob;f=jenkins/jobs/jobs.yaml;h=a1a29f4b0fdaa4730c842736ee7e2ae4e49d06da;hb=HEAD | |
| 13:48:44 | BobBall | which is not easily testable | |
| 13:48:45 | toabctl | mriedem, yes. the test is not executed. | |
| 13:48:58 | toabctl | but then I should reenable the test with my changeset | |
| 13:49:17 | openstackgerrit | Andy McCrae proposed openstack/nova: Allow placement endpoint interface to be set https://review.openstack.org/426163 | |
| 13:49:33 | mriedem | BobBall: ah so i can't use a depends-on like the xenserver ci | |
| 13:49:43 | toabctl | oh. it's on the external CI. so I can not simply change that | |
| 13:49:53 | mriedem | right | |
| 13:50:08 | mriedem | BobBall: we could move that blacklist into the nova tree... | |
| 13:50:09 | toabctl | so I did test that manually on our internal CI... | |
| 13:50:11 | mriedem | then it's branch-specific | |
| 13:50:13 | BobBall | Sorry mriedem, it's not... | |
| 13:50:28 | mriedem | BobBall: like this https://github.com/openstack/nova/blob/master/devstack/tempest-dsvm-lxc-rc | |
| 13:51:04 | mriedem | then that job definition just needs to use the DEVSTACK_GATE_TEMPEST_REGEX variable | |
| 13:51:32 | BobBall | That's nice, but curious because clearly it's not *just* nova implicated with some of the tempest tests - for example there have historically been some tempest tests that only work with libvirt | |
| 13:52:09 | BobBall | i.e. the tests make assumptions about the hypervisor e.g. that it can support qcow2 images made in the test themselves | |
| 13:53:08 | mriedem | right so the tempest defaults are based on the nova defaults, which are libvirt | |
| 13:53:19 | mriedem | if you're not using those defaults, you have to configure stufff | |
| 13:53:34 | mriedem | that doesn't mean we couldn't control the xenproject ci blacklist from within the nova tree | |
| 13:53:42 | BobBall | Indeed - but the case I was thinking about wasn't configurable | |
| 13:55:17 | mriedem | bauzas: rechecked your scheduler stuff, the grenade patch merged before cells v1 broke, but the cells v1 thing is fixed now so all of the rechecks are running | |
| 13:55:36 | bauzas | mriedem: cool, thanks for following that | |
| 13:57:30 | mriedem | oomichi: sdague: johnthetubaguy: want to take a crack at this? i've got a +2 on it https://review.openstack.org/#/c/424759/ | |
| 13:57:43 | mriedem | and the tempest change is approved https://review.openstack.org/#/c/426991/ | |
| 13:58:58 | johnthetubaguy | mriedem: talking a look now, would be good to get that in | |
| 13:59:16 | johnthetubaguy | mriedem: there was a spec to match it too, but thats more a documentation thing at this point I guess? | |