| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-12-07 | |||
| 19:55:28 | mriedem | agreed? | |
| 19:55:42 | jaypipes | works for me. | |
| 19:55:43 | cdent | that sounds right | |
| 19:56:30 | mriedem | ok, +W the | |
| 19:56:31 | mriedem | *then | |
| 19:56:33 | cdent | i agree we’ve not been careful about versions, but I’m not sure it is a huge problem in reality | |
| 19:56:40 | melwitt | mriedem: FYI I added some test coverage here and stacked the remove old quotas code follow up on top https://review.openstack.org/#/c/526270 | |
| 20:00:56 | mriedem | melwitt: comments inline | |
| 20:02:46 | jaypipes | mriedem: reviewed https://review.openstack.org/#/c/525787/. +Wd | |
| 20:03:18 | melwitt | mriedem: I didn't have access to self.flags in fixtures.py, I assumed it's on the base TestCase class. but let me check | |
| 20:04:22 | mriedem | melwitt: you likely have to pass the test's self into the fixture | |
| 20:04:27 | mriedem | like we do in some other fixtures | |
| 20:04:37 | mriedem | i care less about the self.flags thing | |
| 20:04:46 | mriedem | and more about doing the cleanup after the thing you change, and removing the unused CONF in the sample test | |
| 20:04:54 | melwitt | oh, got it. I can do that then | |
| 20:05:04 | mriedem | jaypipes: thanks, replied about the setup thing | |
| 20:05:11 | melwitt | k | |
| 20:08:09 | mriedem | jaypipes: btw, i haven't dealt with that much mox in a long time... | |
| 20:08:29 | mriedem | the strictness with which mox makes sure you hit bdm.save() cost me about 2 hours | |
| 20:08:58 | jaypipes | mriedem: I know, right? :) | |
| 20:09:54 | cdent | good night | |
| 20:12:56 | mriedem | jaypipes: efried: want me to do the doc and nova-status stuff from https://review.openstack.org/#/c/385693/ ? | |
| 20:13:25 | efried | mriedem I would welcome that. Were you planning to do it in isolation or add it somewhere in this series? | |
| 20:13:31 | mriedem | isolation | |
| 20:13:35 | mriedem | i'm not touching that mess | |
| 20:13:36 | efried | Perfect | |
| 20:13:39 | efried | Yeah, you shouldn't. | |
| 20:19:13 | melwitt | I don't know why but using self.flags causes the concurrent test fail for the NoopQuotaDriver | |
| 20:19:36 | mriedem | huh, maybe not global enough? | |
| 20:23:41 | melwitt | oh because self.flags doesn't clear the conf override. weird, I thought it would have to so it works with multiple tests | |
| 20:24:29 | efried | melwitt I have a bug for that. | |
| 20:24:30 | efried | Stand by... | |
| 20:24:47 | james_li_ | Hi nova devs, a quick question: is it possible to attach a volume of another tenant to my server? | |
| 20:25:24 | mriedem | james_li_: depends on the policy configuration, but not by default | |
| 20:25:25 | melwitt | gah, what other channel did I accidentally leave by pressing ctrl-w in the wrong window | |
| 20:25:47 | efried | melwitt Sorry, different bug, probably not related: https://bugs.launchpad.net/oslo.config/+bug/1709728 | |
| 20:25:48 | openstack | Launchpad bug 1709728 in oslo.config "CONF.set_override doesn't alias deprecated opts" [Undecided,Confirmed] | |
| 20:26:13 | openstackgerrit | Eric Berglund proposed openstack/nova master: WIP: PowerVM Driver: vSCSI https://review.openstack.org/526094 | |
| 20:26:44 | james_li_ | mriedem: cool so its possible if policy enables, e.g. admin user? | |
| 20:27:09 | mriedem | james_li_: the default rule is admin_or_owner, | |
| 20:27:13 | mriedem | if that were changed to @ | |
| 20:27:16 | mriedem | then anything goes | |
| 20:27:56 | mriedem | https://docs.openstack.org/nova/latest/configuration/sample-policy.html | |
| 20:28:03 | openstackgerrit | Eric Berglund proposed openstack/nova master: WIP: PowerVM Driver: vSCSI https://review.openstack.org/526094 | |
| 20:28:04 | james_li_ | mriedem: thanks, just wanted to make sure if any code changes are needed for that. | |
| 20:28:09 | mriedem | #"os_compute_api:os-volumes-attachments:create": "rule:admin_or_owner" | |
| 20:28:30 | mriedem | james_li_: shouldn't require code changes | |
| 20:28:36 | mriedem | but policy is a funny thing so you'd have to test it | |
| 20:28:46 | james_li_ | :) | |
| 20:31:51 | melwitt | api samples tests run super ugly under py3 | |
| 20:32:12 | melwitt | lots of warnings emitted to the screen | |
| 20:32:36 | melwitt | and clearing the override didn't seem to help. gdi | |
| 20:33:50 | efried | jaypipes Do we have db constraints or other checks that prevent traits, aggs, and children from existing against a provider UUID that doesn't exist? | |
| 20:34:08 | jaypipes | efried: nope. | |
| 20:34:39 | mriedem | melwitt: just use set_override, it's fine | |
| 20:34:41 | efried | jaypipes So theoretically I could create RP traits first, then create the RP they're associated with? | |
| 20:34:57 | mriedem | and yes on the py35 warnings - i was seeing that in the functional tests earlier today, oslo.contet and oslo.policy warnings | |
| 20:34:57 | efried | Or create an agg with orphan RP UUIDs, then create the RPs associated with those UUIDs? | |
| 20:35:01 | mriedem | we should consider ignoring those | |
| 20:35:35 | jaypipes | efried: yes, manually executing SQL statements. of course, the object layer won't allow you to do that, though. | |
| 20:36:04 | jaypipes | efried: since a TraitNotFound or ResourceProviderNotFound would be raised when attempting to do that via ResourceProvider.set_traits() | |
| 20:36:25 | jaypipes | efried: because there's a lookup of trait ID to names supplied in set_traits() | |
| 20:36:28 | efried | jaypipes oh, okay, phew. So the answer from the perspective of a REST API consumer is that we're strict about that stuff. | |
| 20:36:36 | jaypipes | efried: yes. | |
| 20:36:40 | efried | Good. | |
| 20:36:53 | jaypipes | efried: except for aggregates, which are just UUIDs and we have nothing to "check" against. | |
| 20:37:10 | efried | jaypipes Right, but you can't *associate* a random anonymous UUID with an aggregate? | |
| 20:37:34 | jaypipes | efried: no. but you *can* associated a random UUID to a known resource provider. | |
| 20:37:43 | jaypipes | efried: it's the aggregate UUID we have no way of checking. | |
| 20:37:50 | efried | jaypipes Got it, cool, thanks. | |
| 20:37:55 | jaypipes | pas de probleme | |
| 20:38:40 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Update nova-status and docs for nova-compute requiring placement 1.14 https://review.openstack.org/526505 | |
| 20:38:41 | efried | jaypipes FYI this is coming from a place where I'm refactoring _ensure_resource_provider: In the _create_resource_provider path I can actually skip refresh_aggregate_map | |
| 20:38:44 | mriedem | efried: jaypipes: ^ | |
| 20:38:59 | jaypipes | efried: ack. | |
| 20:39:03 | jaypipes | mriedem: thank you sir. | |
| 20:39:08 | efried | mriedem Thanks | |
| 20:40:34 | mriedem | imacdonn: you can ask but you're going to be hard pressed to find anyone that knows much about libvirt+xen in icehouse in channel right now | |
| 20:40:40 | mriedem | anthonyper is your closest bet | |
| 20:42:03 | imacdonn | yeah, I know ... OK .. so the issue is that nova-compute occassionally gets stuck seemingly in trying to talk to libvirt .. the symptoms are that the resource_tracker no longer reports every minute, and any VM operations that need libvirt fail | |
| 20:42:32 | imacdonn | via guru meditation, I can see that the resource_tracker thread is stuck trying to call libvirt's getLibVersion() | |
| 20:42:51 | mriedem | which is then a call to the hypervisor | |
| 20:42:57 | mriedem | so you're likely deadlocking on something in the hypervisor | |
| 20:42:58 | imacdonn | that call is made through eventlet's thread pooling proxy thingy, which is now holding a lock | |
| 20:43:06 | mriedem | yeah that could also be screwing you | |
| 20:43:16 | mriedem | i'd enable debug logging for libvirt and see if something shows up in there | |
| 20:43:31 | mriedem | or, check to see if things changed around that code since icehouse and see if you need to backport a fix | |
| 20:43:52 | imacdonn | problem is it happens once in a while, and I have like 2k compute nodes ... don't really want to to turn debug on on all of them and wait | |
| 20:44:16 | imacdonn | I've looked around, but not found anything that looks like an obvious related fix | |
| 20:45:03 | imacdonn | I can't tell for sure if it's libvirt hanging on the call, or eventlet getting hung up somehow and not even trying the call | |
| 20:48:43 | mriedem | imacdonn: well, i see this in kilo https://review.openstack.org/#/c/104930/ | |
| 20:49:16 | mriedem | https://review.openstack.org/#/c/104930/12/nova/virt/libvirt/host.py@192 | |
| 20:49:17 | mriedem | so, | |
| 20:49:18 | imacdonn | yeah, that made it "fun" to try to compare bits of of the code to see what might have changed | |
| 20:49:29 | mriedem | keep in mind that anything running in those threads that logs anything could lock you up | |
| 20:49:48 | imacdonn | yes, saw some stuff about that | |
| 20:49:50 | mriedem | so if you have a GMR when things are locked, i'd look for any libvirt driver/host methods in the thread dump, | |
| 20:49:53 | mriedem | and see if those do loging | |
| 20:49:55 | mriedem | *logging | |
| 20:50:22 | imacdonn | GMR is at https://pastebin.com/1jfgdurJ | |