Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-11-21
14:28:19 efried gmann: ^
14:28:49 sean-k-mooney it should be python 3 https://zuul.opendev.org/t/openstack/build/b26bd7fc82d94d4cac97b5481623b629/log/logs/grenade.sh.txt.gz#16522
14:29:02 sean-k-mooney but if it was install on python 2 first
14:29:06 johnsom Well, it looks like it was properly installed in the 3.6 environment given it was a python3 devstack setup. It's just somehow the script is using the py2 openstack command.
14:29:10 sean-k-mooney then the console script would be python 2
14:29:21 johnsom https://zuul.opendev.org/t/openstack/build/323afb9d5fd94f62b0bba4bac6004442/log/logs/pip2-freeze.txt.gz#77
14:30:19 sean-k-mooney oh i know what the issue is
14:30:34 sean-k-mooney the first run install in python 2 right?
14:30:35 gmann yeah, let's recheck as devstack is all py3 now ?
14:30:39 johnsom Well, I guess that explains why it only fails when devstack is set to python3
14:30:48 sean-k-mooney e.g. train would run with python 2
14:30:56 gmann oh yeah
14:31:00 sean-k-mooney then we upgrade to ussuri with python 3
14:31:16 sean-k-mooney and we keep the ocs console script form python 2
14:31:27 sean-k-mooney then we install osc-placmeent in py36
14:31:36 sean-k-mooney which the python2 version wont find
14:31:43 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: Pick NEXT_MIN libvirt/QEMU versions for "V" release https://review.opendev.org/694821
14:31:44 openstackgerrit Kashyap Chamarthy proposed openstack/nova master: libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Ussuri" https://review.opendev.org/695056
14:35:49 sean-k-mooney so we would not see this if we set USE_PYTHON3=True in the grenade job
14:36:03 sean-k-mooney since both version would run on python 3
14:37:07 sean-k-mooney and in a real install the package manager/installer would unistall the python 2 versions when installing the python3 version for ussuri
14:37:40 sean-k-mooney or in container land you would jsut spin up the new python3 only container inplace of the old contianer
14:38:13 sean-k-mooney im pretty sure kolla already move to python 3 contaienr in train for what its worth
14:41:24 openstackgerrit Matt Riedemann proposed openstack/nova master: PoC for using COMPUTE_SAME_HOST_COLD_MIGRATE https://review.opendev.org/695220
14:48:00 mriedem does any of this grenade talk have anything to do with nova/
14:48:02 mriedem ?
14:48:20 mriedem still the resource provider thing or what?
14:52:43 johnsom lol, well, it's the nova scripting in grenade that is failing. That is the tie back to nova.
14:54:08 mriedem i could have sworn at one point in grenade we had some kind of hack where we'd re-install python-openstackclient b/c we went from 2 to 3
14:54:10 mriedem but i'm not finding that
15:15:15 openstackgerrit Matt Riedemann proposed openstack/nova master: Avoid spurious error logging in _get_compute_nodes_in_db https://review.opendev.org/695453
15:17:35 ayoung What does Nova do to modify the kernel command line? It has to happen prior to cloud-init. I can see docs that imply I should be able to do this: glance image-update --property kernel_extra_args="coreos.inst.ignition_url=http://example.com/config.ign " coreos-xx-bootstrap
15:17:59 ayoung but that does not show up when the image boots, and I am guessing I need to pass that through to the instance somehow
15:19:13 sean-k-mooney if you pass a seperate kernel image in addtion to the root image i think we can pass the kernel arges to qemu
15:19:34 sean-k-mooney but in general im not sure how much that featuer is used or tested
15:19:57 sean-k-mooney i do not belive you can use it with just a root image
15:20:10 johnthetubaguy ayoung did you try os_command_line: https://github.com/openstack/nova/blob/1cd5563f2dd2b218db2422397c8aab394d484626/nova/objects/image_meta.py#L463
15:20:43 johnthetubaguy hmm, I am not so sure we do anything with that, ignore me
15:20:44 sean-k-mooney johnthetubaguy: isnt os_command_line jsut for lxc and other containers
15:20:57 sean-k-mooney like openvz
15:21:10 mriedem "The kernel command line to be used by the libvirt driver, instead of the default. For Linux Containers (LXC), the value is used as arguments for initialization. This key is valid only for Amazon kernel, ramdisk, or machine images (aki, ari, or ami)."
15:21:24 johnthetubaguy yeah, my bad
15:21:32 ayoung BTW, this is a pretty good argument for Nova supporting ignition the same way we do cloud-init...it happens earlier in the process
15:22:11 sean-k-mooney its also a good argument for ignition supporting the metadata service :P
15:22:12 ayoung I know that the openshift install, which works via terraform, does something to inject this value. I do not know what that is
15:22:54 ayoung So, I think the metadata service would work. I think what you are saying is that the ignition mech should default to the cloud-init URL, somehow?
15:23:35 ayoung Well, not the URL, but the host, and a separate URL specific to ignition.
15:23:48 sean-k-mooney i was suggesting that ignition could try to hit the metadta url and then load info form it
15:23:53 ayoung Yep
15:24:25 ayoung or a logical shift from it, like: http://meta-data-host/ignition
15:25:01 sean-k-mooney ayoung: anyway the way that was all ment to work was ehn you boot the vm you provide a sperate kernel image with the kernel command line parmater set and then nova woudl use the root iamge and kernel image when booting and pass the kernel args form teh kernel image to qemu
15:25:10 ayoung And Nova/metadata server then would be responsible for multiplexing between the different instances
15:26:00 ayoung I see. I don;t think that the installer (terraform) is doing any of that. But I can reproduce this afternoon and determine what it IS doing
15:35:58 mriedem efried: heh, oops https://review.opendev.org/#/c/694897/
15:36:01 mriedem not sure i/we missed that
15:36:15 mriedem i mean, it's zvm so totally forgettable but otherwise
15:37:58 openstackgerrit Dan Smith proposed openstack/nova master: ZVM: Implement update_provider_tree https://review.opendev.org/694897
15:44:14 aarents Hi dansmith, can you confirm this last upaste is ok for you since your were holding -1 on that https://review.opendev.org/#/c/670000 ? thanks !
15:45:08 dansmith aarents: yeah, I was waiting for CI before, but I'll circle back
15:45:24 dansmith aarents: we probably want to have a few people look at that and sanity check my thinking there
15:45:52 dansmith maybe gibi since he was previously okay with the other fix, at least
15:46:09 dansmith obviously mriedem is always good at everything
15:46:20 aarents yes good idea
15:47:32 mriedem umm, rescue + disk bus = i defer to lyarwood
15:47:56 dansmith eh?
15:48:08 dansmith it's not really rescue related,
15:48:11 mriedem "There is a case during rescue where this value can be mistakenly updated
15:48:11 mriedem to reflect disk bus property of rescue image (hw_disk_bus)."
15:48:17 dansmith rescue is just one place where this side effect screws us
15:48:30 dansmith right, but.. the change is actually more generic
15:48:39 dansmith but yes, lyarwood would be a good person to look also
15:49:08 mriedem my gut feeling on something like this is it fixes one thing and breaks another
15:49:14 mriedem and i don't have the background on that
15:49:26 dansmith mriedem: check out my comments on the earlier set
15:49:43 dansmith mriedem: a patch from gary unceremoniously removed all instance.save()s he thought were unnecessary,
15:49:51 dansmith which removed the one right after this that was there originally,
15:49:57 dansmith so we just almost never save this change
15:49:59 dansmith and also,
15:50:10 dansmith making a change to instance within a "just generate me the xml" method is crazy wrong
15:50:50 dansmith but rescue just happens to tickle things right so we mangle the disk bus, but never save() it back to normal
15:51:06 sean-k-mooney we should not be saving the disk bus back but we shoudl be usign for the rescue xml
15:52:46 sean-k-mooney i dont think there has ever been an expection that the /dev/sd* names for the rescue boot would be the same as when the instnace is booted normally
15:53:06 dansmith sean-k-mooney: read the patches I linked in my analysis earlier
15:53:24 mriedem would have been useful to have that context in the commit message, if just summarized from the big comment in PS1
15:53:39 dansmith sean-k-mooney: the original change from like 2012 was trying to use get_xml as a hook to update the info late after libvirt had chosen defaults
15:54:26 mriedem tl;dr https://review.opendev.org/#/c/119622/ removed the code that would persist this change and yet we can still incorrectly persist it in edge cases incorrectly (like rescue) and we shouldn't be modifying the instance in a _get* method anyway.
15:54:55 sean-k-mooney ok ya we should not
15:55:07 dansmith mriedem: exactly
15:55:15 mriedem so do that, dan can +2 and ill +W
15:55:20 dansmith aarents: ^
15:59:41 aarents dansmith: So I rephrase commit message with more context ?
16:00:13 dansmith aarents: yeah just add all that context in there to make mriedem happy
16:00:23 aarents ok got it !
16:00:58 mgoddard hi mriedem, got a minute to discuss https://review.opendev.org/#/c/684849 ?
16:15:17 mriedem mgoddard: sure
16:15:32 mriedem preface: i have lost a lot of context on that bug and fix
16:16:15 mriedem so i'll likely just abandon my changes and you can move forward
16:16:30 mgoddard do you happen to remember how the RP association becomes stale after your patch
16:16:36 mgoddard I can't work it out from the code
16:17:21 mgoddard https://review.opendev.org/#/c/684849/2/nova/tests/functional/regressions/test_bug_1841481.py@129

Earlier   Later