| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-11-02 | |||
| 18:06:22 | lyarwood | Overall things should remain the same but I'm not entirely sure if we persist the MAC addresses, sean-k-mooney ^ any idea? | |
| 18:07:05 | Zer0Byte | yeah is running cloud init again | |
| 18:07:09 | Zer0Byte | changing the ssh key | |
| 18:08:48 | lyarwood | cloud-init shouldn't regenerate ssh keys if they already exist right? | |
| 18:09:05 | Zer0Byte | mhmm if the machine id change | |
| 18:09:10 | Zer0Byte | trigger execute cloud init agai | |
| 18:09:11 | Zer0Byte | n | |
| 18:09:38 | Zer0Byte | and cloudinit perform ssh-keygen | |
| 18:10:56 | sean-k-mooney[m] | lyarwood: the mac adress comes from the neutron port so it wont change | |
| 18:11:06 | sean-k-mooney[m] | the machine id i guess you mean the one shown in dmidecoe | |
| 18:11:22 | sean-k-mooney[m] | i think that depends on your config but i think its the vms uuid by default | |
| 18:11:51 | sean-k-mooney[m] | https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.sysinfo_serial | |
| 18:14:00 | sean-k-mooney[m] | Zer0Byte: so it would only change if you had set the sysinfo_serial to OS which uses the host /etc/machine-id file , hardware which uses the host uuid form libvirt or auto whcih choses between those | |
| 18:14:50 | sean-k-mooney[m] | Zer0Byte: so with our default config of unique unshleving should not change the guest serial and cloud-init should not run | |
| 18:15:23 | sean-k-mooney[m] | the mac adress should not change either unless you changed it manually in neutron while i was sheleved | |
| 18:17:11 | lyarwood | anyway there's some additional tooling in Xena to refresh connection_info for shutdown instances without the need to shelve and unshelve etc | |
| 18:17:32 | lyarwood | https://docs.openstack.org/nova/latest/cli/nova-manage.html#volume-attachment-commands | |
| 18:17:59 | lyarwood | that's another way to workaround it while we try to work something out in-tree | |
| 18:18:34 | lyarwood | tbh I can't think of a way with the current cinder APIs | |
| 18:19:13 | Zer0Byte | i got to go thanks anyway i will check the option of @sean-k-mooney[m] | |
| 18:20:26 | sean-k-mooney[m] | no worries im offically on pto until tomorrow anyway so not really here today just saw you question while i was checking over my car insurance | |
| 19:01:23 | EugenMayer | Anybody in here uses freezer (successfully?) | |
| 19:16:00 | dansmith | bauzas: when you return: https://blueprints.launchpad.net/nova/+spec/nova-change-default-overcommit-values | |
| 20:10:42 | hyang[m] | Can anyone help to take a look this patch https://review.opendev.org/c/openstack/nova/+/811521 thanks in advance! | |
| 22:31:55 | opendevreview | Merged openstack/nova stable/stein: [stable-only] Pin virtualenv and setuptools https://review.opendev.org/c/openstack/nova/+/813451 | |
| #openstack-nova - 2021-11-04 | |||
| 00:47:12 | opendevreview | Ghanshyam proposed openstack/nova stable/train: DNM: test tempest train-last tag https://review.opendev.org/c/openstack/nova/+/816598 | |
| 08:01:33 | gibi | good morning | |
| 08:03:10 | opendevreview | Balazs Gibizer proposed openstack/nova stable/victoria: Parse alias from domain hostdev https://review.opendev.org/c/openstack/nova/+/816486 | |
| 08:07:39 | opendevreview | Balazs Gibizer proposed openstack/nova stable/xena: Prevent leaked eventlets to send notifications https://review.opendev.org/c/openstack/nova/+/816487 | |
| 08:08:47 | opendevreview | Balazs Gibizer proposed openstack/nova stable/xena: Avoid unbound instance_uuid var during delete https://review.opendev.org/c/openstack/nova/+/816488 | |
| 08:09:33 | gibi | bauzas: the reno for the pps work has landed so you can marke the | |
| 08:09:39 | gibi | https://blueprints.launchpad.net/nova/+spec/qos-minimum-guaranteed-packet-rate | |
| 08:09:43 | gibi | bp as implemented | |
| 08:09:46 | gibi | \o/ | |
| 08:11:02 | opendevreview | Balazs Gibizer proposed openstack/nova master: Enable min pps tempest testing in nova-next https://review.opendev.org/c/openstack/nova/+/811748 | |
| 08:15:45 | lyarwood | gibi++ | |
| 08:16:03 | gibi | lyarwood: thanks for the reviews yestarday | |
| 08:16:15 | lyarwood | np, still catching up | |
| 08:20:46 | gibi | I hope you enjoyed the explanation of the fix for https://launchpad.net/bugs/1946339 . It was a real adventure for me in eventlet/greenlet land. | |
| 08:21:53 | jpodivin | Hi everyone. Has anyone ever encountered: "The placement service for 192.168.24.100:regionOne exists but does not have any supported versions." ? | |
| 08:22:27 | jpodivin | It's hitting master in rdo, but wallaby somehow avoids it. | |
| 08:26:08 | gibi | jpodivin: hi, could you try to simply send a GET request to the root of you placement service? | |
| 08:26:47 | gibi | you should see something like https://paste.opendev.org/show/810375/ | |
| 08:27:00 | jpodivin | gibi: hm, sadly it's a proposed CI job, so the nodes are already down. | |
| 08:27:07 | jpodivin | althought I do have the logs stored | |
| 08:27:25 | gibi | then it would be good to check the placement service logs | |
| 08:28:08 | jpodivin | gibi: anything specific I should look for? | |
| 08:28:28 | gibi | jpodivin: do you see requests resulting in 404 there? | |
| 08:28:38 | EugenMayer | when running nova backup on an image, i see pending operations in the UI. Is there any way to show this tasks and their detailed operations on the cli? glance taks-list and glance image-tasks do not show any pending tasks | |
| 08:29:13 | gibi | we had an issue with apache config not long ago resulted in wrong proxy config for placement https://review.opendev.org/c/openstack/devstack/+/811389 | |
| 08:29:31 | jpodivin | gibi: no, it looks fairly clean, couple of warnings but those seem related to deprecation. https://logserver.rdoproject.org/99/36499/6/check/periodic-tripleo-ci-centos-8-containers-multinode-compute-master-validation/91262ed/logs/subnode-1/var/log/containers/placement/placement.log.txt.gz | |
| 08:31:31 | jpodivin | gibi: found another log, that might be what you meant. Lists a bunch of requests but 404 https://logserver.rdoproject.org/99/36499/6/check/periodic-tripleo-ci-centos-8-containers-multinode-compute-master-validation/91262ed/logs/subnode-1/var/log/containers/httpd/placement/placement_wsgi_access.log.txt.gz | |
| 08:32:33 | jpodivin | Interesting thing is: it starts *after* the error occurs, not before. | |
| 08:32:40 | gibi | yeah, these logs seems to indicate your placement service works correctly and some clients (probably nova) was able to communicate with it | |
| 08:33:23 | gibi | if the error happens after the logs ends then it can be that your placement service was simply stopped / crashed | |
| 08:34:10 | jpodivin | gibi: looking at the time stamps, it looks like the error occurs a approximately one minute before the placement requests log starts . | |
| 08:34:50 | jpodivin | gibi: scrap that. It's actually about 3 seconds before | |
| 08:35:05 | jpodivin | still, before any communication is recorded . | |
| 08:35:25 | gibi | still it means that the placement server is not running when you client tried to read the supported versions from it | |
| 08:40:44 | lyarwood | bauzas / melwitt ; https://review.opendev.org/c/openstack/nova/+/807025 - would you mind hitting this and the fups on top this week? | |
| 08:45:31 | jpodivin | gibi: yes that does seem to be the case | |
| 08:45:50 | jpodivin | gibi: question is: why would that happen on master and not on wallaby. | |
| 09:06:37 | gibi | jpodivin: I have no idea | |
| 09:34:34 | lyarwood | sean-k-mooney: can you hit https://review.opendev.org/c/openstack/nova/+/811716 again when you get a chance | |
| 09:35:43 | EugenMayer | after deploying with kolla and tls, it seems like the GUI based TTY console is still behin to http, not https, while anything else is properly behind tls encryption. The point is, horizon tries to load from https:6080 .. but it fails (no socket). http:6080 has a socket, but the token seems to be invalid there. Any hints what could be wrong, i | |
| 09:35:43 | EugenMayer | assume this is related to the nova service? | |
| 09:36:50 | opendevreview | Lee Yarwood proposed openstack/nova master: WIP configdrive: Move mkisofs_cmd default to mkisofs https://review.opendev.org/c/openstack/nova/+/808921 | |
| 09:36:59 | EugenMayer | when looking at the nova-compute configuration i see https://gist.github.com/EugenMayer/058499029fbd298600a8efa634687c92 | |
| 09:37:52 | bauzas | gibi: done, https://blueprints.launchpad.net/nova/+spec/qos-minimum-guaranteed-packet-rate is now Implemented | |
| 09:37:59 | gibi | bauzas: thanks | |
| 09:38:03 | bauzas | (thanks for noticing me) | |
| 09:38:10 | bauzas | lyarwood: ack, will look | |
| 09:38:13 | EugenMayer | soseems like https://bugzilla.redhat.com/show_bug.cgi?id=1722089 is related | |
| 09:40:12 | opendevreview | Lee Yarwood proposed openstack/nova master: DNM/WIP libvirt: Default x86_64 instances to the q35 machine type https://review.opendev.org/c/openstack/nova/+/816629 | |
| 09:41:04 | kashyap | lyarwood: --^ I'm curious too... :) Thx for posting it | |
| 09:41:07 | lyarwood | EugenMayer: the downstream Red hat bug isn't related | |
| 09:42:14 | EugenMayer | lyarwood thank you. Not sure what i miss then, reading https://docs.openstack.org/nova/xena/admin/remote-console-access.html | |
| 09:42:31 | EugenMayer | a little hard to understand what kolla did and did not | |
| 09:45:33 | EugenMayer | this is what i have in nova-vnc https://gist.github.com/EugenMayer/82528fcfca6e22b818f865852606f28c - currently kind of double posting since i'am not sure this is a nova 'issue' or a kolla 'configuration' issue or a kolla 'deployment bug' | |
| 09:47:03 | lyarwood | EugenMayer: I'd think that's a kolla config bug tbh | |
| 09:47:44 | lyarwood | the URL returned from n-api is https right? | |
| 09:47:55 | lyarwood | when you do a console url show? | |
| 09:48:29 | EugenMayer | lyarwood how would i do that? | |
| 09:51:56 | lyarwood | EugenMayer: https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/console-url.html#console-url-show openstack console url show $instance | |
| 09:54:46 | EugenMayer | lyarwood the URL looks good. HTTPS, right fqdn, 60800 | |
| 09:55:55 | EugenMayer | telent on that port works, but i assume it talks http | |
| 09:56:18 | EugenMayer | wget returns 'GnuTLS: Error in the pull function. | |
| 09:56:18 | EugenMayer | Unable to establish SSL connection.' | |
| 09:56:37 | lyarwood | kk then it's the nova-novncproxy service that isn't configured correctly and thus a Kolla bug | |
| 09:59:52 | EugenMayer | lyarwood thank you. Already looking at the configuration. I assume the certificates are not bound properly | |
| 10:02:20 | EugenMayer | lyarwood https://gist.github.com/EugenMayer/fba4eb20a49ccd717ba70f38188a8e1e - i'am using officially signed certificates. Maybe missing https://docs.openstack.org/nova/xena/admin/remote-console-access.html#novnc-proxy-server-configuration /etc/pki/libvirt-vnc/server-cert.pem or something? | |
| 10:04:49 | lyarwood | that's to encrypt the connection between the proxy service and libvirtd | |
| 10:05:03 | lyarwood | I think your issue is between the user and the proxy service right? | |
| 10:10:28 | EugenMayer | yes | |
| 10:29:08 | EugenMayer | lyarwood any hints where to start | |
| 10:29:56 | lyarwood | EugenMayer: I'm not sure how Kolla configures things tbh but I'd start with the config file associated with the nova-novncproxy service itself | |
| 10:30:15 | lyarwood | EugenMayer: see if it differs from nova-api etc that are working with tls already | |
| 10:30:19 | EugenMayer | that's what i posted lyarwood - the problem is, not sure what to expect there | |
| 10:34:03 | opendevreview | Merged openstack/nova stable/xena: Store old_flavor already on source host during resize https://review.opendev.org/c/openstack/nova/+/810911 | |
| 10:36:08 | lyarwood | EugenMayer: https://github.com/openstack/nova/blob/909cfc76369b94b026cf42b86fb5a310dce21a8c/nova/cmd/baseproxy.py#L48-L87 - so looking at the code it's using configurables like ssl_only and cert | |