| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-03 | |||
| 12:05:30 | openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Add node_uuid field to Destination object https://review.openstack.org/649532 | |
| 12:05:31 | openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Add in_tree field to RequestGroup object https://review.openstack.org/649534 | |
| 12:05:31 | openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: Pass node uuid to new Destination.node_uuid https://review.openstack.org/649533 | |
| 12:05:32 | openstackgerrit | Tetsuro Nakamura proposed openstack/nova master: node_uuid from RequestSpec to ResourceRequest https://review.openstack.org/649535 | |
| 12:08:28 | shilpasd | hi: observed that in multinode setup, instance data files created at all nodes (controller + compute in my case) at 'instances_path', why is this so? | |
| 12:09:40 | openstackgerrit | Merged openstack/nova master: Style corrections for privsep usage. https://review.openstack.org/648615 | |
| 12:09:51 | openstackgerrit | Merged openstack/nova master: Remove mox in unit/network/test_neutronv2.py (4) https://review.openstack.org/574106 | |
| 12:10:00 | openstackgerrit | Merged openstack/nova master: Remove mox in unit/network/test_neutronv2.py (5) https://review.openstack.org/574110 | |
| 12:18:24 | sean-k-mooney | o/ | |
| 12:19:00 | sean-k-mooney | are there any known gate issues related to installing hacking package | |
| 12:19:22 | sean-k-mooney | i think the failure im seeing was just an intermitent netowrk issue but | |
| 12:19:41 | sean-k-mooney | before i recheck https://review.openstack.org/#/c/649409/ i just said i would ask | |
| 12:21:06 | sean-k-mooney | ill jsut recheck as everything else passed | |
| 12:22:53 | sean-k-mooney | melwitt: when https://review.openstack.org/#/c/649409/ merges ill cherry pick it back to stable/stein | |
| 12:25:56 | sean-k-mooney | tonyb: speaking of stable backport i fixed the indentation in https://review.openstack.org/#/c/635075/2..3 | |
| 12:38:55 | efried | mriedem: Right (forbidden traits never worked) | |
| 12:39:51 | alex_xu | eandersson: yea, tags is better for just lable instance, and searching http://specs.openstack.org/openstack/nova-specs/specs/newton/implemented/tag-instances.html | |
| 12:43:00 | sean-k-mooney | efried: it worked in a flavor right just not in an image? | |
| 12:43:28 | sean-k-mooney | efried: or is it just totally borked for everything on the nova sise | |
| 12:43:32 | sean-k-mooney | side | |
| 12:43:57 | efried | sean-k-mooney: it didn't work in flavor, but it was supposed to. | |
| 12:44:12 | efried | sean-k-mooney: it doesn't work in image, but we never implemented that, so that part's not surprising. | |
| 12:44:14 | sean-k-mooney | oh ok so it only works at the placement api level | |
| 12:44:19 | efried | right | |
| 12:44:26 | efried | well, now it works from flavor too. | |
| 12:44:33 | sean-k-mooney | ah ok | |
| 12:44:34 | efried | because fixed | |
| 12:44:39 | sean-k-mooney | :) | |
| 12:44:42 | efried | probably stein backport-worthy. | |
| 12:44:53 | efried | but probably not RC-worthy | |
| 12:45:05 | efried | not sure | |
| 12:45:13 | sean-k-mooney | it was "intoduced" in rocky right? | |
| 12:45:55 | sean-k-mooney | yep https://github.com/openstack/nova-specs/blob/master/specs/rocky/implemented/placement-forbidden-traits.rst | |
| 12:45:58 | efried | sean-k-mooney: The placement side was a year ago. | |
| 12:46:30 | efried | trying to find the nova side | |
| 12:47:36 | sean-k-mooney | i think it was all covered by the same spec for rocky | |
| 12:47:43 | shilpasd | efried: observed that in multinode setup, instance data files created at all nodes (controller + compute in my case) at 'instances_path', is it correct behavior? | |
| 12:49:12 | efried | sean-k-mooney: https://blueprints.launchpad.net/nova/+spec/forbidden-traits-in-nova different spec | |
| 12:49:50 | efried | shilpasd: sounds like a libvirt question. sean-k-mooney or stephenfin, y'all have that answer off the top? | |
| 12:50:00 | sean-k-mooney | efried: assuming this was added in rocky then its not an rc candiate since the regession was not in stien but i would agrue the fix should be backported to both stable/stien and stable/rocky as it was just a bug in the original implemenation and not a new feature | |
| 12:50:24 | sean-k-mooney | efried: shilpasd i did not anser because i did not know. | |
| 12:50:59 | sean-k-mooney | if by instance data path you mean under /var/lib/libvirt* then yes | |
| 12:51:02 | efried | sean-k-mooney: https://review.openstack.org/#/c/561677/ <== rocky. So yeah. | |
| 12:51:05 | sean-k-mooney | i think that is correct | |
| 12:51:24 | shilpasd | efried: sean-k-mooney: okay, will check with stephenfin, 'instances_path = /opt/stack/data/nova/instances' | |
| 12:51:49 | openstackgerrit | Matthew Booth proposed openstack/nova master: systemd detection result caching nit fixes https://review.openstack.org/649229 | |
| 12:51:50 | sean-k-mooney | efried: so ya not an RC candiate but assuming the fix is non invasive it should be a backprot candiate | |
| 12:52:09 | sean-k-mooney | shilpasd: is this a devstack setup | |
| 12:53:33 | shilpasd | sean-k-mooney: yes | |
| 12:53:43 | sean-k-mooney | if so the /opt/stack/data directory is where we sotre image and cinder voluems and other perstnet data form the openstack services | |
| 12:53:56 | sean-k-mooney | so i would not be surpiesd if the instacne root disk is also stored there | |
| 12:54:20 | sean-k-mooney | i have never really checked however. | |
| 12:54:41 | shilpasd | sean-k-mooney: but in multinode case, it was stored at both node in my case, at controller side and at compute side | |
| 12:55:08 | sean-k-mooney | shilpasd: it wont be sotre in both localtions it will be copied if you do a migration | |
| 12:55:42 | sean-k-mooney | e.g. we wont create the isntace root disk on all nodes in the multinode devstack cloud | |
| 12:56:03 | openstackgerrit | Eric Fried proposed openstack/nova stable/stein: Adding tests to demonstrate bug #1821824 https://review.openstack.org/649600 | |
| 12:56:04 | openstackgerrit | Eric Fried proposed openstack/nova stable/stein: Fix bug preventing forbidden traits from working https://review.openstack.org/649601 | |
| 12:56:04 | shilpasd | sean-k-mooney: yes, but before migration i am verifying 'instances_path' and observed that instance files are already there | |
| 12:56:04 | openstack | bug 1821824 in OpenStack Compute (nova) stein "Forbidden traits in flavor properties don't work" [High,Confirmed] https://launchpad.net/bugs/1821824 | |
| 12:56:30 | shilpasd | sean-k-mooney: IMO it should not be the case, as you said during migration it should copy | |
| 12:56:56 | sean-k-mooney | if you are doing a block migration it will copy them yes | |
| 12:57:35 | sean-k-mooney | you can also mount /opt/stack/data/nova on nfs to avoid this but you would have to do that manually | |
| 12:57:38 | openstackgerrit | Eric Fried proposed openstack/nova stable/rocky: Adding tests to demonstrate bug #1821824 https://review.openstack.org/649602 | |
| 12:57:39 | openstackgerrit | Eric Fried proposed openstack/nova stable/rocky: Fix bug preventing forbidden traits from working https://review.openstack.org/649603 | |
| 12:57:39 | openstack | bug 1821824 in OpenStack Compute (nova) stein "Forbidden traits in flavor properties don't work" [High,In progress] https://launchpad.net/bugs/1821824 - Assigned to Eric Fried (efried) | |
| 12:58:01 | sean-k-mooney | shilpasd: are you tryign to debug somthing locally or in the gate | |
| 12:58:17 | sean-k-mooney | or just understand how it works | |
| 12:58:23 | shilpasd | sean-k-mooney: tried with NFS + multinode, there saw insatnce data file created at all 3 places in my case, at NFS shared location, at controller and at compute node. | |
| 12:58:42 | sean-k-mooney | ah its not create at all 3 locations | |
| 12:59:06 | sean-k-mooney | the data is stored on the nfs share and its just mounted on the contoler and compute | |
| 12:59:24 | sean-k-mooney | there is one copy of the data but its acceable via nfs form multiple locations | |
| 12:59:36 | shilpasd | sean-k-mooney: yes me to wondered, and here during evacuation getting error 'libvirtError: internal error: process exited while connecting to monitor.', 'ERROR oslo_messaging.rpc.server Is another process using the image?' | |
| 13:00:10 | sean-k-mooney | shilpasd: what version of openstack are you deploying | |
| 13:00:46 | shilpasd | sean-k-mooney: stein | |
| 13:01:02 | sean-k-mooney | when we do an evacuate we assume the guest on the donw node is stoped | |
| 13:01:09 | sean-k-mooney | if its not its not safe to evacuate | |
| 13:01:14 | shilpasd | openstack 3.18.0 | |
| 13:01:23 | sean-k-mooney | did you stop the vm? | |
| 13:01:44 | shilpasd | purposefully stopping n-cpu by service-force-down | |
| 13:02:00 | sean-k-mooney | that will not stop the vm | |
| 13:02:27 | shilpasd | yes but evacuation happens in this case also | |
| 13:02:57 | sean-k-mooney | yes and the if you use force down you are required to check that the vms are stopped on the host | |
| 13:03:45 | sean-k-mooney | force does was added specficaly for operator that had external monitorin that could determin the host had failed and would stop all reunning guest before seting it | |
| 13:04:19 | shilpasd | ok, will check that, but IMO error i am getting is because of same data files already available and during evacuation the said error ''ERROR oslo_messaging.rpc.server Is another process using the image?'' | |
| 13:04:33 | sean-k-mooney | https://specs.openstack.org/openstack/nova-specs/specs/liberty/implemented/mark-host-down.html#use-cases | |
| 13:05:02 | shilpasd | ok, will check after VM down, but original question remains, why insatnce data files created at all 3 places NFS, controller and compute? | |
| 13:05:16 | shilpasd | is it correct behaviour? | |
| 13:05:22 | sean-k-mooney | yes | |
| 13:05:34 | sean-k-mooney | assumign that the instance directory is on nfs | |
| 13:05:53 | sean-k-mooney | what does mount show on the contoler or compute node | |
| 13:06:32 | sean-k-mooney | we have special logic that detect if the instace directory is on nfs and skips copying the disk in that cases | |
| 13:06:33 | shilpasd | same shared location | |
| 13:06:55 | shilpasd | can you please help me where exatly this logic is | |
| 13:07:23 | shilpasd | i have checked def _create_image() of libvirt | |
| 13:08:26 | openstackgerrit | Jared Winborne proposed openstack/nova master: Leave brackets on Ceph IP addresses for libguestfs https://review.openstack.org/649405 | |
| 13:10:08 | sean-k-mooney | shilpasd: https://github.com/openstack/nova/blob/1554d35834a474514f827449bd7d4f1d2f0af1d6/nova/virt/libvirt/driver.py#L6611-L6641 | |
| 13:11:19 | sean-k-mooney | shilpasd: the issue here however is not related to this check. | |
| 13:11:36 | KH-Jared | I'm just going to blame my long lines on trusting my IDE too much. Its pep8 warning was at 120 characters instead of 79, whoops | |
| 13:12:07 | sean-k-mooney | if you use force host down, you are required to ensure the vm is not running before you call evacuate. that is the cause of the error you hit | |
| 13:13:20 | shilpasd | sean-k-mooney: yes, thanks for this input, will check with VM down, and will debug more the given refrence code, and get back to you on the same later | |