| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-25 | |||
| 18:12:26 | sean-k-mooney | well it requires you too chagne it right | |
| 18:12:28 | dansmith | no | |
| 18:12:31 | sean-k-mooney | no | |
| 18:12:46 | dansmith | we have many fields which intentionally have different nullability for the db and object | |
| 18:12:52 | dansmith | including one we just did for the quota stuff | |
| 18:13:00 | sean-k-mooney | allowing an object to have a nullable field that was previousl not nullable does not require the db to allow it to be null? | |
| 18:13:08 | sean-k-mooney | ohok | |
| 18:13:12 | dansmith | not necessarily | |
| 18:13:18 | dansmith | totally depends on the thing | |
| 18:15:54 | jhinman | sean: that doesn't affect the instance's socket perms, does it? | |
| 18:16:14 | sean-k-mooney | jhinman: its not he socket perms that is the issue | |
| 18:16:41 | sean-k-mooney | qemu is createing the socket so it will be the owner and will have read and write permissions | |
| 18:16:47 | sean-k-mooney | ovs is runnign as root | |
| 18:16:56 | sean-k-mooney | so the issue is the permission on the directory | |
| 18:17:09 | sean-k-mooney | nova need read write permission to create the unix socket | |
| 18:17:54 | sean-k-mooney | 777 should give all users and groups read wite permission to create files there | |
| 18:20:39 | jhinman | but that script has been there all along | |
| 18:20:58 | sean-k-mooney | jhinman: we bind mount the host /run into both the nova-libvirt and ovs vswtichd container | |
| 18:21:30 | sean-k-mooney | can you check the permission on the host | |
| 18:21:32 | sean-k-mooney | ls -al /run/openvswitch/ | |
| 18:22:04 | jhinman | ok, will do. co-worker just told me she has it working on master branch, so something got fixed | |
| 18:22:39 | sean-k-mooney | jhinman: perhaps but none of this code has change in well over 8 month and most is 2+ years old | |
| 18:23:29 | sean-k-mooney | jhinman: that said i wrote most of the ovs-dpdk code in kolla so im not surpised if there are bugs :) | |
| 18:24:06 | jhinman | host has 777 on /run/openvswitch | |
| 18:26:02 | jhinman | I thought the log is saying that it's creating a socket in the VM | |
| 18:26:24 | sean-k-mooney | no its creating a socket for the vm. not in the vm. | |
| 18:26:59 | jhinman | a socket in nova-compute? | |
| 18:27:00 | sean-k-mooney | its created on the host by qemu i recent release. it was previously created by ovs and connected to by qemu in old releases | |
| 18:27:32 | sean-k-mooney | the socket wil be create in the nova-libvirt container which also has the same directory bind monted shared form the host | |
| 18:27:47 | sean-k-mooney | jhinman: https://github.com/openstack/kolla-ansible/blob/stable/rocky/ansible/roles/nova/defaults/main.yml#L16 | |
| 18:28:14 | jhinman | ok. I'll figure it out. thanks for your help! | |
| 18:29:24 | sean-k-mooney | sorry i couldnt help more. i shoudl really test that code again but i have not been working on kolla for about 18 months so i havent really been aroudn to take care of it | |
| 18:30:47 | sean-k-mooney | jhinman: if you have issue im also on the #openstack-kolla channel so feel free to ping me there not to bother the nova folks. if its not specific feel free to ping me here | |
| 18:53:40 | openstackgerrit | sean mooney proposed openstack/nova-specs master: add spec for image metadata prefiltering https://review.openstack.org/647578 | |
| 18:56:49 | mnaser | would it be ok to add https://review.openstack.org/#/c/645352/ to the rc2 list? and https://review.openstack.org/#/c/641907/ ? | |
| 18:59:30 | sean-k-mooney | mnaser: oh that the fix for the null issue you were looking at last week? | |
| 19:01:22 | mnaser | sean-k-mooney: correct, the first one is that, the second one is Oslo.service doing a full restart of the service instead of reload, which breaks some assumptions nova does | |
| 19:01:29 | mnaser | which means reloading to refresh info leaves you in a broken state | |
| 19:02:35 | sean-k-mooney | mnaser: right i brifly skimed that thread on the mailing list | |
| 19:02:54 | sean-k-mooney | oh so that was how this was getting to None in the first place | |
| 19:03:39 | mnaser | sean-k-mooney: yeah I could see it in the DB going from "NULL" (sql null) to "null" (son null) | |
| 19:03:40 | sean-k-mooney | or rather how self._bdm_obj.connection_info was | |
| 19:03:42 | mnaser | json* | |
| 19:08:19 | sean-k-mooney | mnaser: by the way on the olso service thing my intiall reaction was make oslos reload not call stop and start but i have no idea what other effect that would have | |
| 19:08:42 | sean-k-mooney | mnaser: but that is more have sane names for things then because it definetly the right thing to do | |
| 19:10:34 | openstackgerrit | sean mooney proposed openstack/nova-specs master: re add for train https://review.openstack.org/647581 | |
| 19:15:08 | openstackgerrit | sean mooney proposed openstack/nova-specs master: forward ported form stien https://review.openstack.org/647582 | |
| 19:17:42 | sean-k-mooney | mriedem: do ^ need to go back through full review or do can they be fast approved. i litrally just copied them updated the history footer? i know we are still focusing on RC2 but im hoping we can land that code before the ptg or m1 | |
| 19:18:20 | sean-k-mooney | mriedem: i know we want to talk about openlab for the numa testing at the ptg but just said i would ask | |
| 19:19:47 | efried | sean-k-mooney: If they were approved in stein, and they're verbatim (except the footer and s/stein/train/i) then they can be fast approved. | |
| 19:20:24 | sean-k-mooney | efried: ya they are | |
| 19:20:29 | efried | sean-k-mooney: However, I would request that you make some reference to the actual blueprint being proposed in the commit title and message body. | |
| 19:20:42 | melwitt | mnaser: are they regressions? I think we're only doing regression fixes for RC2 | |
| 19:20:46 | efried | sean-k-mooney: those titles are going to kill me | |
| 19:20:57 | sean-k-mooney | sure :) | |
| 19:21:04 | efried | thanks | |
| 19:21:07 | sean-k-mooney | i was just going for dinner and wanted to push something | |
| 19:21:15 | efried | cool | |
| 19:21:21 | efried | if you need to run, I can edit and fast approve. | |
| 19:21:27 | sean-k-mooney | ill fix them later. should i add them to the metting aggenda? | |
| 19:21:45 | melwitt | sean-k-mooney: instructions, you need the Previously-approved in the commit message https://specs.openstack.org/openstack/nova-specs/readme.html#previously-approved-specifications | |
| 19:22:02 | sean-k-mooney | melwitt ah perfect thanks | |
| 19:22:16 | efried | sean-k-mooney: You mean for the PTG? Unless there's something more to be discussed on them, I wouldn't think so. | |
| 19:22:47 | efried | sean-k-mooney: I guess they will come up in a "what do we prioritize for train" discussion, but that'll be fed by whatever's in the train approved blueprint list at the time anyway. | |
| 19:22:56 | sean-k-mooney | efried: no i wasnt sure if i was ment to add fast approavl spec to the nova team meeting agenda | |
| 19:23:01 | sean-k-mooney | ill just do what the doc says | |
| 19:23:30 | efried | oh, yeah, that won't be necessary. I (or someone) will push them as soon as they conform to what melwitt linked above. | |
| 19:24:01 | sean-k-mooney | efried: cool in that case ill fix them up and upload them later tonight | |
| 19:24:08 | efried | sean-k-mooney: ++ | |
| 19:25:00 | mnaser | melwitt: no both are long standing upgrade bugs afaik so yeah | |
| 19:29:09 | mriedem | mnaser: i've got a todo to try and recreate that first one with an in-tree functional test, which shouldn't take me long | |
| 19:29:33 | mriedem | idk about that oslo.service one - note that would require a release freeze exception on oslo.service and a minimum required version of oslo.service bump in nova (maybe that'd be optional though) | |
| 19:32:19 | mnaser | well, I think we need to come to consensus on if the issue is in Oslo service or nova. | |
| 19:32:37 | mnaser | Imho it’s Oslo service cause then what’s the point of sighup if you’re gonna stop and start... | |
| 19:32:53 | dansmith | mnaser: I've lost context here, but is there anything nova can do to _not_ be fully torn down and reset? | |
| 19:33:07 | dansmith | because I thought not, based on how we get started and managed from oslo | |
| 19:34:16 | dansmith | as it is, a "systemctl restart nova-compute" is much better than the behavior we're getting from HUP, IMHO | |
| 19:34:37 | mnaser | dansmith: based on my understanding it looks like Oslo service never really provides an API to handle HUP signals. | |
| 19:35:01 | dansmith | mnaser: you mean it grabs HUP itself, but doesn't do what a HUP signal should really do | |
| 19:35:09 | dansmith | and, if so, I agree :) | |
| 19:35:48 | mnaser | Exactly. It calls reload but only after calling stop and start after so that’s a bit useless at that point. | |
| 19:36:00 | dansmith | right | |
| 19:36:21 | mnaser | I think the most portable potential fix is adding a keyword argument to the stop() function call which includes sighup=bool | |
| 19:36:26 | dansmith | that doesn't seem like a reasonable status quo to me, which means I think it's an oslo.service bug, not a "nova should be doing something different" scenario | |
| 19:36:36 | efried | I agree with that | |
| 19:36:49 | mnaser | Yes. I think Nova is doing the right thing as well. | |
| 19:36:53 | dansmith | efried: agree with which? | |
| 19:37:05 | mnaser | Nothing except reload() should be called on SIGHUP. | |
| 19:37:07 | efried | dansmith: `it's an oslo.service bug, not a "nova should be doing something different" scenario` | |
| 19:37:12 | dansmith | efried: ack | |
| 19:37:21 | efried | And we've got stuff documented that is supposed to happen to nova on HUP | |
| 19:37:27 | dansmith | right | |
| 19:37:34 | efried | which currently does happen, but is worthless because it destroys the process | |
| 19:37:44 | dansmith | yarp | |
| 19:38:12 | efried | stuff I have the confidence to say in IRC, but not on the ML | |
| 19:38:21 | efried | but I have been watching that thread with interest | |
| 19:38:31 | mnaser | So my review here solved this but “changed” existing behaviour ... https://review.openstack.org/#/c/641907 | |
| 19:38:42 | mnaser | So is it a bug fix or behaviour change. Up for debate. | |
| 19:39:04 | efried | it can be both | |
| 19:39:27 | dansmith | the only argument not to do that would be to keep the existing behavior because it's always been broken, but that leaves us unable to resolve our issue | |