| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-04-03 | |||
| 12:44:26 | efried | well, now it works from flavor too. | |
| 12:44:33 | sean-k-mooney | ah ok | |
| 12:44:34 | efried | because fixed | |
| 12:44:39 | sean-k-mooney | :) | |
| 12:44:42 | efried | probably stein backport-worthy. | |
| 12:44:53 | efried | but probably not RC-worthy | |
| 12:45:05 | efried | not sure | |
| 12:45:13 | sean-k-mooney | it was "intoduced" in rocky right? | |
| 12:45:55 | sean-k-mooney | yep https://github.com/openstack/nova-specs/blob/master/specs/rocky/implemented/placement-forbidden-traits.rst | |
| 12:45:58 | efried | sean-k-mooney: The placement side was a year ago. | |
| 12:46:30 | efried | trying to find the nova side | |
| 12:47:36 | sean-k-mooney | i think it was all covered by the same spec for rocky | |
| 12:47:43 | shilpasd | efried: observed that in multinode setup, instance data files created at all nodes (controller + compute in my case) at 'instances_path', is it correct behavior? | |
| 12:49:12 | efried | sean-k-mooney: https://blueprints.launchpad.net/nova/+spec/forbidden-traits-in-nova different spec | |
| 12:49:50 | efried | shilpasd: sounds like a libvirt question. sean-k-mooney or stephenfin, y'all have that answer off the top? | |
| 12:50:00 | sean-k-mooney | efried: assuming this was added in rocky then its not an rc candiate since the regession was not in stien but i would agrue the fix should be backported to both stable/stien and stable/rocky as it was just a bug in the original implemenation and not a new feature | |
| 12:50:24 | sean-k-mooney | efried: shilpasd i did not anser because i did not know. | |
| 12:50:59 | sean-k-mooney | if by instance data path you mean under /var/lib/libvirt* then yes | |
| 12:51:02 | efried | sean-k-mooney: https://review.openstack.org/#/c/561677/ <== rocky. So yeah. | |
| 12:51:05 | sean-k-mooney | i think that is correct | |
| 12:51:24 | shilpasd | efried: sean-k-mooney: okay, will check with stephenfin, 'instances_path = /opt/stack/data/nova/instances' | |
| 12:51:49 | openstackgerrit | Matthew Booth proposed openstack/nova master: systemd detection result caching nit fixes https://review.openstack.org/649229 | |
| 12:51:50 | sean-k-mooney | efried: so ya not an RC candiate but assuming the fix is non invasive it should be a backprot candiate | |
| 12:52:09 | sean-k-mooney | shilpasd: is this a devstack setup | |
| 12:53:33 | shilpasd | sean-k-mooney: yes | |
| 12:53:43 | sean-k-mooney | if so the /opt/stack/data directory is where we sotre image and cinder voluems and other perstnet data form the openstack services | |
| 12:53:56 | sean-k-mooney | so i would not be surpiesd if the instacne root disk is also stored there | |
| 12:54:20 | sean-k-mooney | i have never really checked however. | |
| 12:54:41 | shilpasd | sean-k-mooney: but in multinode case, it was stored at both node in my case, at controller side and at compute side | |
| 12:55:08 | sean-k-mooney | shilpasd: it wont be sotre in both localtions it will be copied if you do a migration | |
| 12:55:42 | sean-k-mooney | e.g. we wont create the isntace root disk on all nodes in the multinode devstack cloud | |
| 12:56:03 | openstackgerrit | Eric Fried proposed openstack/nova stable/stein: Adding tests to demonstrate bug #1821824 https://review.openstack.org/649600 | |
| 12:56:04 | openstack | bug 1821824 in OpenStack Compute (nova) stein "Forbidden traits in flavor properties don't work" [High,Confirmed] https://launchpad.net/bugs/1821824 | |
| 12:56:04 | shilpasd | sean-k-mooney: yes, but before migration i am verifying 'instances_path' and observed that instance files are already there | |
| 12:56:04 | openstackgerrit | Eric Fried proposed openstack/nova stable/stein: Fix bug preventing forbidden traits from working https://review.openstack.org/649601 | |
| 12:56:30 | shilpasd | sean-k-mooney: IMO it should not be the case, as you said during migration it should copy | |
| 12:56:56 | sean-k-mooney | if you are doing a block migration it will copy them yes | |
| 12:57:35 | sean-k-mooney | you can also mount /opt/stack/data/nova on nfs to avoid this but you would have to do that manually | |
| 12:57:38 | openstackgerrit | Eric Fried proposed openstack/nova stable/rocky: Adding tests to demonstrate bug #1821824 https://review.openstack.org/649602 | |
| 12:57:39 | openstack | bug 1821824 in OpenStack Compute (nova) stein "Forbidden traits in flavor properties don't work" [High,In progress] https://launchpad.net/bugs/1821824 - Assigned to Eric Fried (efried) | |
| 12:57:39 | openstackgerrit | Eric Fried proposed openstack/nova stable/rocky: Fix bug preventing forbidden traits from working https://review.openstack.org/649603 | |
| 12:58:01 | sean-k-mooney | shilpasd: are you tryign to debug somthing locally or in the gate | |
| 12:58:17 | sean-k-mooney | or just understand how it works | |
| 12:58:23 | shilpasd | sean-k-mooney: tried with NFS + multinode, there saw insatnce data file created at all 3 places in my case, at NFS shared location, at controller and at compute node. | |
| 12:58:42 | sean-k-mooney | ah its not create at all 3 locations | |
| 12:59:06 | sean-k-mooney | the data is stored on the nfs share and its just mounted on the contoler and compute | |
| 12:59:24 | sean-k-mooney | there is one copy of the data but its acceable via nfs form multiple locations | |
| 12:59:36 | shilpasd | sean-k-mooney: yes me to wondered, and here during evacuation getting error 'libvirtError: internal error: process exited while connecting to monitor.', 'ERROR oslo_messaging.rpc.server Is another process using the image?' | |
| 13:00:10 | sean-k-mooney | shilpasd: what version of openstack are you deploying | |
| 13:00:46 | shilpasd | sean-k-mooney: stein | |
| 13:01:02 | sean-k-mooney | when we do an evacuate we assume the guest on the donw node is stoped | |
| 13:01:09 | sean-k-mooney | if its not its not safe to evacuate | |
| 13:01:14 | shilpasd | openstack 3.18.0 | |
| 13:01:23 | sean-k-mooney | did you stop the vm? | |
| 13:01:44 | shilpasd | purposefully stopping n-cpu by service-force-down | |
| 13:02:00 | sean-k-mooney | that will not stop the vm | |
| 13:02:27 | shilpasd | yes but evacuation happens in this case also | |
| 13:02:57 | sean-k-mooney | yes and the if you use force down you are required to check that the vms are stopped on the host | |
| 13:03:45 | sean-k-mooney | force does was added specficaly for operator that had external monitorin that could determin the host had failed and would stop all reunning guest before seting it | |
| 13:04:19 | shilpasd | ok, will check that, but IMO error i am getting is because of same data files already available and during evacuation the said error ''ERROR oslo_messaging.rpc.server Is another process using the image?'' | |
| 13:04:33 | sean-k-mooney | https://specs.openstack.org/openstack/nova-specs/specs/liberty/implemented/mark-host-down.html#use-cases | |
| 13:05:02 | shilpasd | ok, will check after VM down, but original question remains, why insatnce data files created at all 3 places NFS, controller and compute? | |
| 13:05:16 | shilpasd | is it correct behaviour? | |
| 13:05:22 | sean-k-mooney | yes | |
| 13:05:34 | sean-k-mooney | assumign that the instance directory is on nfs | |
| 13:05:53 | sean-k-mooney | what does mount show on the contoler or compute node | |
| 13:06:32 | sean-k-mooney | we have special logic that detect if the instace directory is on nfs and skips copying the disk in that cases | |
| 13:06:33 | shilpasd | same shared location | |
| 13:06:55 | shilpasd | can you please help me where exatly this logic is | |
| 13:07:23 | shilpasd | i have checked def _create_image() of libvirt | |
| 13:08:26 | openstackgerrit | Jared Winborne proposed openstack/nova master: Leave brackets on Ceph IP addresses for libguestfs https://review.openstack.org/649405 | |
| 13:10:08 | sean-k-mooney | shilpasd: https://github.com/openstack/nova/blob/1554d35834a474514f827449bd7d4f1d2f0af1d6/nova/virt/libvirt/driver.py#L6611-L6641 | |
| 13:11:19 | sean-k-mooney | shilpasd: the issue here however is not related to this check. | |
| 13:11:36 | KH-Jared | I'm just going to blame my long lines on trusting my IDE too much. Its pep8 warning was at 120 characters instead of 79, whoops | |
| 13:12:07 | sean-k-mooney | if you use force host down, you are required to ensure the vm is not running before you call evacuate. that is the cause of the error you hit | |
| 13:13:20 | shilpasd | sean-k-mooney: yes, thanks for this input, will check with VM down, and will debug more the given refrence code, and get back to you on the same later | |
| 13:13:25 | sean-k-mooney | KH-Jared: ya the 79 column limmit is particlaly annoying because no ide defualt to 79. some defult to 80 but even pycharm does not defualt to 79 and its a python focused ide | |
| 13:15:15 | KH-Jared | I can get behind it though, the code has always been extremely nice to read, I assume it adds to that. I also have it fixed in my ide now, so hopefully won't see a pep8 failure again in the future | |
| 13:16:08 | sean-k-mooney | i have been working on openstack for the better part of 6 years and i still dont write pep8 complient code by default | |
| 13:16:26 | sean-k-mooney | but i run fast8 on it most of the time before i push | |
| 13:16:44 | sean-k-mooney | KH-Jared: you are aware of the fast8 env | |
| 13:16:50 | sean-k-mooney | tox -e fast8 | |
| 13:16:53 | KH-Jared | i am now | |
| 13:17:08 | sean-k-mooney | it runs pep8 but just on your patched files | |
| 13:17:13 | sean-k-mooney | not on all of nova | |
| 13:17:15 | sean-k-mooney | is way fater | |
| 13:17:17 | artom | sean-k-mooney, whoa, that exists? | |
| 13:17:21 | sean-k-mooney | *faster | |
| 13:17:30 | sean-k-mooney | artom: yes... | |
| 13:17:43 | artom | I've been hacking something like it with $(tox -e pep8 `git show --name-only | grep ^nova`) | |
| 13:18:09 | sean-k-mooney | artom: stephenfin added it in pike | |
| 13:18:16 | artom | ... | |
| 13:18:20 | KH-Jared | this was my first change on a large project, not even just openstack, all testing I've done has been much smaller by comparison. I'm going to be using flake8 more often atleast | |
| 13:18:21 | artom | \o/ | |
| 13:18:22 | sean-k-mooney | its also in a few other repos at this point | |
| 13:18:44 | efried | jaypipes: Do we still have one RT per ironic node? | |
| 13:19:23 | mriedem | gibi_off: do you know why this was made into a warning? https://github.com/openstack/nova/blob/b33fa1c054ba4b7d4e789aa51250ad5c8325da2d/nova/scheduler/client/report.py#L1880 we hit that a lot in normal resizes: https://bugs.launchpad.net/nova/+bug/1822917 | |
| 13:19:24 | openstack | Launchpad bug 1822917 in OpenStack Compute (nova) ""Overwriting current allocation" warnings in logs during move operations although there are no failures" [Undecided,In progress] - Assigned to Takashi NATSUME (natsume-takashi) | |
| 13:20:42 | mriedem | efried: there is one RT per nova-compute service, | |
| 13:20:47 | mriedem | and the RT has a dict of compute nodes | |