| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-05-03 | |||
| 17:12:40 | mriedem | exarlos: np, thanks and good luck | |
| 17:20:35 | moshele | mriedem: hi, what are the chances to approve this pike backport https://review.openstack.org/#/c/563501/ ? the packport wan't trivial at all | |
| 17:24:47 | mriedem | moshele: that enables the nova side of a neutron feature added in pike correct? | |
| 17:25:15 | mriedem | and the change on master was originally reverted https://review.openstack.org/#/c/518833/ | |
| 17:25:26 | mriedem | master/queens | |
| 17:27:09 | moshele | meridem: the first version was revert because it broken heat, but the second one was merged https://review.openstack.org/#/c/519066/ | |
| 17:28:18 | mriedem | sure, but i'm still not sure if we should backport something to nova pike to enable a feature in neutron pike | |
| 17:28:34 | mriedem | afk for lunch | |
| 17:30:18 | moshele | meridem: this is the feature enablement in pike https://review.openstack.org/#/c/398265/. the other patch was a bug fix (https://review.openstack.org/#/c/519066/ ) which was introduce by something I change in neutron | |
| 17:31:36 | moshele | merdem: this is the neutron change https://review.openstack.org/#/c/499203/ | |
| 17:31:53 | moshele | mriedem: this is the neutron change https://review.openstack.org/#/c/499203/ | |
| 17:32:43 | moshele | mriedem: ok I will try to catch you after lunchת bon appetit! | |
| 17:44:51 | exarlos | Thanks again, ciao for now. | |
| 17:56:18 | Tahvok | Hey guys! | |
| 17:58:05 | Tahvok | I'm having some boot issues with kvm (using ocata): After creating an instance, I configured some volumes on it, and after shutdown, I cannot bring it up, it's getting stuck in seabios with 'no bootable device' message. If I remove all other volumes I've created, it manages to boot, but fails later as my configuration needs the volumes available | |
| 17:58:45 | Tahvok | So I've checked the bootindex configuration, and it looks like it's completely fine, bootindex is set only on the bootable volume. | |
| 17:58:52 | Tahvok | I should also mention that I'm using ceph | |
| 18:00:21 | mriedem | Tahvok: do you have this https://review.openstack.org/#/c/561613/ ? | |
| 18:00:31 | mriedem | just released in 15.1.1 | |
| 18:00:42 | Tahvok | Nova,version=15.1.1 | |
| 18:01:21 | Tahvok | Let me check the file just to be sure | |
| 18:03:00 | Tahvok | So, I've just checked this, and not I do not have it | |
| 18:06:04 | melwitt | if you do a 'virsh dumpxml' on the instance, you might see the boot_index set incorrectly on your bootable volume (it should be 0). and if it's not, you need the bug fix from 5.1.1 | |
| 18:06:52 | melwitt | this was the bug https://bugs.launchpad.net/nova/+bug/1729584 | |
| 18:06:52 | openstack | Launchpad bug 1729584 in OpenStack Compute (nova) queens "boot from volume + configdrive with virtio-scsi broken (regression)" [High,Fix committed] - Assigned to Artom Lifshitz (notartom) | |
| 18:08:29 | frickler | mriedem: melwitt: removed the -2, will do a closer review tomorrow | |
| 18:08:51 | melwitt | cool, thanks frickler | |
| 18:09:09 | Tahvok | melwitt: yes, it's set as 1 and not 0 | |
| 18:09:31 | Tahvok | I'm restarting nova-compute now, to check the patch | |
| 18:10:08 | Tahvok | And it's working! | |
| 18:10:28 | melwitt | sweet | |
| 18:10:33 | Tahvok | I wonder why it says that I have nova 15.1.1 when I definitely do not have this patch | |
| 18:11:13 | melwitt | hm, yeah, it should be in there | |
| 18:11:40 | melwitt | 5.1.1 was released yesterday, very recent | |
| 18:12:28 | Tahvok | So I definitely do not have this | |
| 18:12:49 | Tahvok | As we rerun ansible a week ago | |
| 18:13:34 | melwitt | do you build packages yourself from stable/ocata? or where did you get the packages? | |
| 18:13:45 | Tahvok | I'm using openstack-ansible project | |
| 18:14:29 | melwitt | okay. I'm not familiar with how it works but I'm guessing maybe it builds packages off stable as the "next version" which would have been 5.1.1 | |
| 18:14:51 | melwitt | that is, if you were to run it today it would give you 5.1.2, just a guess | |
| 18:15:19 | Tahvok | Yes, it's uses hashes from git | |
| 18:15:34 | Tahvok | Ok, so that might be why I see 5.1.1 | |
| 18:15:39 | eandersson | Random question but why does the scheduler need to log every time it does an instance sync? | |
| 18:15:57 | mriedem | eandersson: need to? | |
| 18:16:02 | Tahvok | I will request to bump the hashes so I'll get this fix 'officially' | |
| 18:16:04 | mriedem | because if thinks you care | |
| 18:16:09 | eandersson | :D | |
| 18:16:11 | mriedem | *it | |
| 18:16:17 | eandersson | It's just with 800+ nodes it gets a bit spammy :D | |
| 18:16:19 | melwitt | Tahvok: ah, k. yeah. so when we released 5.1.1, that means it was tagged as 15.1.1 and pushed to pypi etc as that version | |
| 18:16:24 | mriedem | i had a patch for oslo.log at one point to be able to drop stuff like that in the scheduler to AUDIT log | |
| 18:16:26 | mriedem | below DEBUG | |
| 18:16:43 | mriedem | eandersson: is it debug today? | |
| 18:16:53 | eandersson | INFO | |
| 18:16:56 | melwitt | Tahvok: this was the release patch. I'm not sure if/how openstack-ansible consumes that https://review.openstack.org/564044 | |
| 18:17:02 | eandersson | but we are still on mitaka (so might have changed) | |
| 18:17:22 | mriedem | eandersson: which specific message? | |
| 18:17:40 | mriedem | Successfully synced instances from host ? | |
| 18:18:04 | mriedem | yeah those should probably be debug | |
| 18:18:14 | eandersson | Yea | |
| 18:18:27 | mriedem | the "received <operation> from unknown host" is probably ok at INFO | |
| 18:18:42 | mriedem | but all the other "OH HI I DID SOMETHING I"M SUPPOSED TO DO!!!" is debug | |
| 18:18:45 | mriedem | *should be debug | |
| 18:18:48 | eandersson | I can throw up a quick commit on gerrit | |
| 18:18:52 | mriedem | ack | |
| 18:18:58 | mriedem | just don't get any on me | |
| 18:20:00 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Create volume attachment during boot from volume in compute https://review.openstack.org/541420 | |
| 18:20:02 | mriedem | melwitt: this time with 10x more bdmturducken ^ call in the next 20 minutes and we'll double your bdms | |
| 18:20:31 | melwitt | and the peasants rejoiced | |
| 18:21:02 | Tahvok | melwitt: mriedem thanks a lot for helping with the issue! And didn't know what was going on... | |
| 18:21:16 | Tahvok | We've had it for a few months now... | |
| 18:21:21 | mriedem | i can't say that's it, but it sounds right | |
| 18:21:34 | mriedem | yeah it was a regression in ocata i believe | |
| 18:21:51 | melwitt | np, I'd been chasing that bug for a long time to get the fix out | |
| 18:49:58 | mriedem | so uh, i don't think this is ever going to be true https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L3196 | |
| 18:50:06 | mriedem | is it possible to reboot a rescued instance? | |
| 18:50:09 | mriedem | the api doesn't allow it | |
| 18:50:39 | mriedem | https://github.com/openstack/nova/blob/master/nova/compute/vm_states.py#L69-L72 | |
| 18:51:15 | mriedem | is there some other place in nova where we internally reboot an instance when it's rescued? | |
| 18:52:49 | melwitt | I know not | |
| 18:53:13 | mriedem | git blame history here i go | |
| 18:54:01 | mriedem | https://github.com/openstack/nova/commit/39ffe806cdfe97721fee5e48c8591c4c3b52766b | |
| 18:54:22 | openstackgerrit | Merged openstack/nova master: support multiple member_of qparams https://review.openstack.org/561315 | |
| 18:54:25 | mriedem | https://github.com/openstack/nova/blob/39ffe806cdfe97721fee5e48c8591c4c3b52766b/nova/compute/api.py#L1702 | |
| 18:54:32 | mriedem | wtf, you couldn't reboot a rescue vm | |
| 18:55:57 | melwitt | what? that patch is about the vm_state, just making sure it stays "rescued" after the reboot. right? | |
| 18:56:01 | mriedem | https://review.openstack.org/#/q/topic:bug/1170237+(status:open+OR+status:merged) | |
| 18:56:18 | mriedem | yeah, but if you look at the reboot api code in that same hash, we don't allow rebooting a rescued vm | |
| 18:56:28 | melwitt | oh :\ | |
| 18:56:35 | mriedem | i mean, we don't allow rebooting an instance with vm_state=RESCUE | |
| 18:56:37 | mriedem | https://review.openstack.org/#/q/topic:bug/1170237+(status:open+OR+status:merged) | |
| 18:56:49 | mriedem | looks like some rax/xen thing | |
| 18:56:54 | mriedem | that likely never made the api change upstream | |
| 18:56:57 | mriedem | gd rax | |
| 18:57:25 | melwitt | guh, weird. so we have patches that say "now you can reboot a rescued instance" but you can't because the API kicks you out | |
| 18:57:38 | mriedem | the rax api code probably lets you | |
| 18:57:41 | melwitt | right | |
| 18:57:51 | mriedem | where was that one guy that still works at rax? | |
| 18:58:01 | mriedem | the guy always in korea.. | |
| 18:58:08 | melwitt | tbh, I don't know whether we're suppose to be able to reboot rescued instances. I'm not that familiar with the "rescue" function | |