| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-02-13 | |||
| 18:10:33 | sean-k-mooney | cdent: also i dont know | |
| 18:10:37 | openstackgerrit | Jim Rollenhagen proposed openstack/nova master: ironic: check fresh data when sync_power_state doesn't line up https://review.openstack.org/636699 | |
| 18:11:06 | cdent | sean-k-mooney: yeah, I was confused by the choice of lock id | |
| 18:11:10 | cdent | (too) | |
| 18:14:32 | sean-k-mooney | cdent: i have not looked at the code but is your requestion related to the use of the uuid for synconisation or is detach ungaurded and your asking should it be | |
| 18:15:31 | cdent | sean-k-mooney: I'm asking if it should be. In k8s-on-openstack environment there are issues happening with volume attachements getting into a weird state | |
| 18:15:32 | openstackgerrit | garyk proposed openstack/nova master: Lock detach_volume https://review.openstack.org/636701 | |
| 18:15:44 | cdent | that's ^ the proposed fix | |
| 18:16:58 | sean-k-mooney | cdent: lyarwood and mdbooth are usually who i ask when it comes to volumn/storage things | |
| 18:17:26 | cdent | yeah, me too, but I'm hoping they've already stepped awy, keeping reasonable hours for this timezone, unlike you and me | |
| 18:17:44 | lyarwood | I'm around, just | |
| 18:18:04 | lyarwood | we already lock within the libvirt driver, I assume this is for VMWare? | |
| 18:18:26 | cdent | lyarwood: in the compute manager itself | |
| 18:18:31 | cdent | not the virtdrivers | |
| 18:18:35 | cdent | see the patch | |
| 18:18:57 | lyarwood | cdent: right, and I assume the issues you're seeing are with the underlying detach/attach on the virt drivers? | |
| 18:19:08 | lyarwood | cdent: or is it with the attachments themselves in c-api? | |
| 18:19:43 | cdent | lyarwood: I'm short on details. At the moment I'm just feeling around trying to gather info. I wasn't present at the discovery of the issue. | |
| 18:19:54 | lyarwood | kk np | |
| 18:20:36 | openstackgerrit | garyk proposed openstack/nova master: Lock detach_volume https://review.openstack.org/636701 | |
| 18:22:09 | cdent | lyarwood: as far as I can tell whatever the client is (some k8s thing) is going async on the volume delete and instance delete | |
| 18:22:36 | openstackgerrit | Matt Riedemann proposed openstack/nova master: api-ref: mark os-cells as deprecated https://review.openstack.org/636708 | |
| 18:23:13 | lyarwood | cdent: yeah and as vmware isn't using os-brick you don't get any of the locking within it, adding a lock in the compute layer for detach seems reasonable tbh. | |
| 18:23:13 | mriedem | cdent: i seem to remember mdbooth having a change to lock detach_volume on instance.uuid | |
| 18:23:44 | cdent | mriedem: you mean in progress? | |
| 18:23:49 | mriedem | might be abandoned by now | |
| 18:24:56 | mriedem | https://review.openstack.org/#/c/430213/ | |
| 18:24:57 | mriedem | bingo | |
| 18:25:01 | sean-k-mooney | lyarwood: well as cdent pointed out we already lock in the compute manager for attach | |
| 18:25:29 | mriedem | cdent: there you go - want me to restore and you can rebase? | |
| 18:25:39 | lyarwood | sean-k-mooney: right, I'm agreeing | |
| 18:26:16 | sean-k-mooney | lyarwood: yep i know. | |
| 18:26:30 | sean-k-mooney | mriedem: is there any reason to restor it vs https://review.openstack.org/#/c/636701/2 | |
| 18:26:47 | sean-k-mooney | the two are identical | |
| 18:26:52 | mriedem | how about because it came first? | |
| 18:27:06 | cdent | thanks mriedem I looked, but not that far back | |
| 18:27:50 | sean-k-mooney | ok i gues garyk could take over rebaseing the old patch | |
| 18:29:00 | cdent | since it is in merge conflict, how about we just booth as a co-author on the clean one, and add the older bug too? | |
| 18:29:42 | mriedem | that works too... | |
| 18:29:47 | mriedem | i've duplicated the bug already | |
| 18:29:55 | mriedem | sec | |
| 18:30:32 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Lock detach_volume https://review.openstack.org/636701 | |
| 18:32:42 | mriedem | comment inline | |
| 18:32:47 | mriedem | i'm not sure we need to hold the lock while doing the db query | |
| 18:32:47 | cdent | damn dood, I think I'm just gonna let you do everything ;) | |
| 18:32:53 | cdent | yeah | |
| 18:33:43 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Lock detach_volume https://review.openstack.org/636701 | |
| 18:34:19 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Lock detach_volume https://review.openstack.org/636701 | |
| 18:34:21 | mriedem | there god | |
| 18:34:23 | mriedem | coding in gerrit ui | |
| 18:35:21 | cdent | Is there any chance that get_by_volume_and_instance itself can race? | |
| 18:35:48 | openstackgerrit | Vlad Gusev proposed openstack/nova stable/queens: Handle unicode characters in migration params https://review.openstack.org/636678 | |
| 18:37:02 | mriedem | worst case is you'd get a not found error from the db query | |
| 18:53:00 | openstackgerrit | Merged openstack/nova master: Change nova-next job to run with python3 https://review.openstack.org/634739 | |
| 18:53:14 | openstackgerrit | Merged openstack/nova master: Adding cross refs for config options in scheduler filter guide https://review.openstack.org/636635 | |
| 19:01:16 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove cells v1 (for the most part) from the docs https://review.openstack.org/636718 | |
| 19:30:41 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: New objects to transmit NUMA config from dest to source https://review.openstack.org/634827 | |
| 19:30:42 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Use live_migration_claim() to check dest resources https://review.openstack.org/634606 | |
| 19:30:42 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: [WIP] Introduce live_migration_claim() https://review.openstack.org/635669 | |
| 19:30:43 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: update NUMA-related XML on the source https://review.openstack.org/635229 | |
| 19:30:43 | openstackgerrit | Artom Lifshitz proposed openstack/nova master: LM: Make dest send NUMAMigrateData to the source https://review.openstack.org/634828 | |
| 19:46:42 | aspiers | what's the easiest way to set the logger to DEBUG level in functional tests? | |
| 19:49:32 | cdent | if you're based off the nova base class, you probably already have it, but if not self.flag('debug', 'True') might do it. I can't remember for sure | |
| 19:49:54 | dansmith | there's an environment variable | |
| 19:50:06 | aspiers | ^^^ that's the answer I was hoping for ;-) | |
| 19:50:09 | melwitt | aspiers: OS_DEBUG=1 | |
| 19:50:10 | cdent | there's a fixture that will swallow a lot of stuff too | |
| 19:50:16 | dansmith | yeah that^ | |
| 19:50:21 | aspiers | awesome, thanks! :-D | |
| 19:50:29 | melwitt | have fun | |
| 19:50:48 | aspiers | found it in nova/tests/fixtures.py now | |
| 19:51:11 | melwitt | OS_DEBUG=1 tox -efunctional test_stuff | |
| 19:51:58 | melwitt | that will show you log level debug and less during a test run | |
| 19:52:05 | aspiers | cdent: I could maybe answer that if I understood your answer, but I fear it's too sophisticated for me ;-) | |
| 19:52:33 | aspiers | melwitt: yep the env var is working great, although I'm running stestr directly in order to run individual groups of tests quicker | |
| 19:52:40 | melwitt | smart | |
| 19:52:41 | aspiers | going through tox is slow :-/ | |
| 19:52:44 | cdent | aspiers: I think I was answering "I want to manipulate log handling in a functional test I'm making" not "I want to see the debug logs" | |
| 19:52:57 | aspiers | cdent: ah yeah, thought it might be something like that :) | |
| 19:53:12 | aspiers | sorry for not being clear enough in the original q | |
| 19:53:22 | cdent | no worries, worked out in the end | |
| 19:54:23 | aspiers | melwitt: shall I add OS_DEBUG to HACKING.rst? | |
| 19:55:59 | dansmith | aspiers: wouldn't you rather keep it secret and seem smart the next time a newbie comes along? | |
| 19:56:38 | aspiers | dansmith: tempting, but I think it's even more satisfying to be able to RTFH somebody by pointing them to docs I wrote myself ;-p | |
| 19:56:54 | dansmith | aspiers: okay as long as shame is involved either way | |
| 19:57:02 | aspiers | that way I can appear smart *and* conceited | |
| 19:57:05 | aspiers | exactly | |
| 19:57:30 | melwitt | haha, nice | |
| 19:57:44 | aspiers | after all I only went into coding in order to be able to shame others :-p | |
| 19:58:36 | melwitt | that's the right motivation | |
| 19:58:49 | aspiers | glad we're on the same page XD | |
| 19:59:17 | melwitt | and yeah, it would be a good addition to HACKING.rst. and also our contributor doc/ | |
| 20:00:38 | aspiers | melwitt: under https://docs.openstack.org/nova/latest/contributor/testing.html or a new page? | |
| 20:00:52 | aspiers | presumably https://docs.openstack.org/nova/latest/contributor/index.html#testing isn't a good place for it | |
| 20:01:20 | melwitt | lemme see, I think I saw a place | |
| 20:02:33 | melwitt | I was looking at https://docs.openstack.org/nova/latest/contributor/development-environment.html#running-unit-tests but yeah, similarly doesn't have the same level of detail that HACKING.rst does | |
| 20:03:08 | aspiers | I could just add a link from there to HACKING? | |
| 20:03:21 | melwitt | that also looks really old :\ | |
| 20:03:30 | aspiers | Yeah | |
| 20:07:58 | melwitt | aspiers: I think a link to HACKING for nova-specific test running notes would be a good fit there on that last page. because the existing doc is a link to the general openstack-wide test running guide | |