| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-21 | |||
| 14:01:32 | gmann | sean-k-mooney: fixing osc might take time with release etc until octavia job can install it from source. | |
| 14:01:54 | sean-k-mooney | do we know what cause the broken pipes? | |
| 14:02:03 | gmann | no. | |
| 14:02:21 | haleyb | sean-k-mooney: would adding osc-placement to requirements in octavia fix it as well? | |
| 14:02:53 | sean-k-mooney | i think mriedem said list does not currently support --name | |
| 14:04:02 | sean-k-mooney | i was looking at the greand job logs by the way gmann haleyb do you have the link to the octavia job | |
| 14:04:33 | johnsom | haleyb Yes, that is the error output you are seeing. Installing osc-placement should fix it. | |
| 14:04:49 | gmann | sean-k-mooney: https://review.opendev.org/#/c/693486/ | |
| 14:04:50 | johnsom | https://www.irccloud.com/pastebin/PrDd7zG6/ | |
| 14:05:08 | openstackgerrit | Merged openstack/nova stable/pike: Delete instance_id_mappings record in instance_destroy https://review.opendev.org/684658 | |
| 14:05:21 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Ussuri" https://review.opendev.org/695056 | |
| 14:05:42 | sean-k-mooney | ah right yes "'resource provider list -f value' is not an openstack command" | |
| 14:05:44 | kashyap | stephenfin: ^^ Fixed the functional test (forgot to run `tox -e functional-36`, bad me) | |
| 14:05:53 | johnsom | haleyb The question is which devstack plugin has the missing requirement | |
| 14:05:55 | sean-k-mooney | is becasue osc-placement is not installed | |
| 14:05:59 | gmann | johnsom: haleyb it is parsing of command failing not about osc command is need more installation etc | |
| 14:06:03 | kashyap | stephenfin: Thanks for the earlier review :-) | |
| 14:06:53 | kashyap | Uh, seems like I need a rebase... | |
| 14:07:04 | sean-k-mooney | johnsom: i guess the ocatavia one | |
| 14:07:17 | gmann | thus command we need to adjust to get the first RP- https://github.com/openstack/grenade/blob/fad62595bff3ae55b5b428a5ea00a9a168390fd2/projects/60_nova/resources.sh#L57 | |
| 14:07:30 | sean-k-mooney | but honestly it might be better for devstack to install osc-placemetn if placement is installed | |
| 14:07:32 | johnsom | sean-k-mooney Octavia devstack plugin isn't running that command. | |
| 14:07:40 | gmann | command i mean parsing logic | |
| 14:07:51 | sean-k-mooney | oh ok | |
| 14:08:32 | johnsom | It is openstack/grenade | |
| 14:08:44 | johnsom | projects/60_nova/resources.sh | |
| 14:08:49 | johnsom | line 57 | |
| 14:08:51 | sean-k-mooney | gmann: im confused on the octaiva patch you are linink to grenade | |
| 14:09:05 | sean-k-mooney | is this a but there are other fialng jobs too | |
| 14:09:14 | sean-k-mooney | *failing jobs | |
| 14:09:27 | sean-k-mooney | are ye only looking at the grenade one at the moment | |
| 14:10:02 | gmann | yeah grenade one only i checked | |
| 14:10:39 | sean-k-mooney | well we could have grenade install it or as i said we could have devstack install it if placemnt is installed | |
| 14:11:12 | sean-k-mooney | grenade is using it so its resonable for it to install its own depencies | |
| 14:11:20 | johnsom | Yeah. It's odd that grenade doesn't have a requirements.txt though it obvious has requirements in it's scripts | |
| 14:11:46 | sean-k-mooney | well grenade is not a python porject | |
| 14:12:39 | sean-k-mooney | its almost all bash so you would not pip install it | |
| 14:13:23 | gmann | osc-placement installation is not the issue here. | |
| 14:13:26 | johnsom | Ha, yeah, I just noticed that. I guess you know now how much time I have looked at grenade.... lol | |
| 14:14:09 | johnsom | gmann Yes it is. If that is not installed the first output of OSC is "openstack:" which the script tries to parse. | |
| 14:15:31 | sean-k-mooney | right as your irccloud link show the message is "openstack: 'resource provider list -f value' is not an openstack command. See 'openstack --help'." | |
| 14:15:47 | johnsom | Yep | |
| 14:17:13 | gmann | https://zuul.opendev.org/t/openstack/build/323afb9d5fd94f62b0bba4bac6004442/log/logs/grenade.sh.txt.gz#13753 | |
| 14:18:45 | kashyap | Is a rebase really necessary here? - https://review.opendev.org/#/c/695056/ | |
| 14:20:48 | sean-k-mooney | gmann: intersting | |
| 14:25:29 | johnsom | I wonder if it is using the py2 python-openstackclient | |
| 14:26:24 | efried | johnsom: I had thought so too, but similar commands just above that seem to be working fine | |
| 14:26:58 | johnsom | I see most of OSC also installed in the py2 environment there. | |
| 14:27:35 | efried | ohhh | |
| 14:27:47 | efried | the working commands above that aren't mucking with providers | |
| 14:27:59 | efried | so yeah, it's probably a matter of osc-placement being installed on the wrong py version. | |
| 14:28:19 | efried | gmann: ^ | |
| 14:28:49 | sean-k-mooney | it should be python 3 https://zuul.opendev.org/t/openstack/build/b26bd7fc82d94d4cac97b5481623b629/log/logs/grenade.sh.txt.gz#16522 | |
| 14:29:02 | sean-k-mooney | but if it was install on python 2 first | |
| 14:29:06 | johnsom | Well, it looks like it was properly installed in the 3.6 environment given it was a python3 devstack setup. It's just somehow the script is using the py2 openstack command. | |
| 14:29:10 | sean-k-mooney | then the console script would be python 2 | |
| 14:29:21 | johnsom | https://zuul.opendev.org/t/openstack/build/323afb9d5fd94f62b0bba4bac6004442/log/logs/pip2-freeze.txt.gz#77 | |
| 14:30:19 | sean-k-mooney | oh i know what the issue is | |
| 14:30:34 | sean-k-mooney | the first run install in python 2 right? | |
| 14:30:35 | gmann | yeah, let's recheck as devstack is all py3 now ? | |
| 14:30:39 | johnsom | Well, I guess that explains why it only fails when devstack is set to python3 | |
| 14:30:48 | sean-k-mooney | e.g. train would run with python 2 | |
| 14:30:56 | gmann | oh yeah | |
| 14:31:00 | sean-k-mooney | then we upgrade to ussuri with python 3 | |
| 14:31:16 | sean-k-mooney | and we keep the ocs console script form python 2 | |
| 14:31:27 | sean-k-mooney | then we install osc-placmeent in py36 | |
| 14:31:36 | sean-k-mooney | which the python2 version wont find | |
| 14:31:43 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: Pick NEXT_MIN libvirt/QEMU versions for "V" release https://review.opendev.org/694821 | |
| 14:31:44 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Ussuri" https://review.opendev.org/695056 | |
| 14:35:49 | sean-k-mooney | so we would not see this if we set USE_PYTHON3=True in the grenade job | |
| 14:36:03 | sean-k-mooney | since both version would run on python 3 | |
| 14:37:07 | sean-k-mooney | and in a real install the package manager/installer would unistall the python 2 versions when installing the python3 version for ussuri | |
| 14:37:40 | sean-k-mooney | or in container land you would jsut spin up the new python3 only container inplace of the old contianer | |
| 14:38:13 | sean-k-mooney | im pretty sure kolla already move to python 3 contaienr in train for what its worth | |
| 14:41:24 | openstackgerrit | Matt Riedemann proposed openstack/nova master: PoC for using COMPUTE_SAME_HOST_COLD_MIGRATE https://review.opendev.org/695220 | |
| 14:48:00 | mriedem | does any of this grenade talk have anything to do with nova/ | |
| 14:48:02 | mriedem | ? | |
| 14:48:20 | mriedem | still the resource provider thing or what? | |
| 14:52:43 | johnsom | lol, well, it's the nova scripting in grenade that is failing. That is the tie back to nova. | |
| 14:54:08 | mriedem | i could have sworn at one point in grenade we had some kind of hack where we'd re-install python-openstackclient b/c we went from 2 to 3 | |
| 14:54:10 | mriedem | but i'm not finding that | |
| 15:15:15 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Avoid spurious error logging in _get_compute_nodes_in_db https://review.opendev.org/695453 | |
| 15:17:35 | ayoung | What does Nova do to modify the kernel command line? It has to happen prior to cloud-init. I can see docs that imply I should be able to do this: glance image-update --property kernel_extra_args="coreos.inst.ignition_url=http://example.com/config.ign " coreos-xx-bootstrap | |
| 15:17:59 | ayoung | but that does not show up when the image boots, and I am guessing I need to pass that through to the instance somehow | |
| 15:19:13 | sean-k-mooney | if you pass a seperate kernel image in addtion to the root image i think we can pass the kernel arges to qemu | |
| 15:19:34 | sean-k-mooney | but in general im not sure how much that featuer is used or tested | |
| 15:19:57 | sean-k-mooney | i do not belive you can use it with just a root image | |
| 15:20:10 | johnthetubaguy | ayoung did you try os_command_line: https://github.com/openstack/nova/blob/1cd5563f2dd2b218db2422397c8aab394d484626/nova/objects/image_meta.py#L463 | |
| 15:20:43 | johnthetubaguy | hmm, I am not so sure we do anything with that, ignore me | |
| 15:20:44 | sean-k-mooney | johnthetubaguy: isnt os_command_line jsut for lxc and other containers | |
| 15:20:57 | sean-k-mooney | like openvz | |
| 15:21:10 | mriedem | "The kernel command line to be used by the libvirt driver, instead of the default. For Linux Containers (LXC), the value is used as arguments for initialization. This key is valid only for Amazon kernel, ramdisk, or machine images (aki, ari, or ami)." | |
| 15:21:24 | johnthetubaguy | yeah, my bad | |
| 15:21:32 | ayoung | BTW, this is a pretty good argument for Nova supporting ignition the same way we do cloud-init...it happens earlier in the process | |
| 15:22:11 | sean-k-mooney | its also a good argument for ignition supporting the metadata service :P | |
| 15:22:12 | ayoung | I know that the openshift install, which works via terraform, does something to inject this value. I do not know what that is | |
| 15:22:54 | ayoung | So, I think the metadata service would work. I think what you are saying is that the ignition mech should default to the cloud-init URL, somehow? | |
| 15:23:35 | ayoung | Well, not the URL, but the host, and a separate URL specific to ignition. | |
| 15:23:48 | sean-k-mooney | i was suggesting that ignition could try to hit the metadta url and then load info form it | |
| 15:23:53 | ayoung | Yep | |