| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-06-09 | |||
| 18:18:57 | artom | Hrmm, I wonder if my fix could stiff have value | |
| 18:21:26 | artom | I guess not, since we'll need to manually update requirements anyways? | |
| 18:21:54 | artom | I did it by just adding `python -m mypy --install-types` to tools/mypywrap.sh | |
| 18:23:31 | lyarwood | Yeah I think stephenfin tried that earlier | |
| 18:23:40 | lyarwood | but it wasn't enough in the gate for some reason | |
| 18:24:12 | artom | I guess because lower-constraints.txt? | |
| 18:42:59 | cz3 | This is probably a trivial question... I wanted to import Nova modules into interactive Python shell, but I cannot figure out how to load the config from default nova.conf. Is `CONF = nova.conf.CONF` supposed to handle this? | |
| 18:43:54 | cz3 | I wanted to play around with objects module through interactive shell but the lack of loaded config is blocking me | |
| 18:45:45 | artom | cz3, never tried it myself, but looking at the code, yeah, import nova.conf; CONF = nova.conf.CONF should do it | |
| 18:49:26 | cz3 | yeah, that's what I tried... doesn't seem to work. looks like it is still trying to use the defaults. I assume that because it tries to look up things in sqlite instead of mysql that I have in nova.conf | |
| 18:51:00 | artom | Maybe it's not finding the config file, and using the defaults... | |
| 18:52:30 | artom | In fact, if you're importing it via Python (and not via running a Python executable on the command line), I'm not even sure how it finds the config file to use... | |
| 18:54:21 | artom | I *think* you want to do CONF = nova.conf.CONF(project='nova') in such a case... | |
| 18:54:29 | artom | And then it'll look for nova.conf in the "normal" places | |
| 18:56:06 | cz3 | yes, I will poke around the nova.conf.CONF, maybe it needs some additional parameters when it is imported into interactive shell | |
| 18:56:29 | sean-k-mooney | cz3: you need to initalise the conf form the file too | |
| 18:56:40 | sean-k-mooney | if you just import it it will get the default form code | |
| 18:57:42 | sean-k-mooney | we do this https://github.com/openstack/nova/blob/master/nova/config.py#L94-L98 | |
| 18:57:53 | sean-k-mooney | where conf is CONF = nova.conf.CONF | |
| 19:02:02 | sean-k-mooney | all the command lets initalise it here https://github.com/openstack/nova/blob/d64edd3da2336a5c7c8f69cced45272cbaf638a9/nova/cmd/status.py#L374 | |
| 19:02:08 | sean-k-mooney | well liek ^ | |
| 19:03:24 | sean-k-mooney | so "import nova.conf; from nova import config; CONF = nova.conf.CONF; config.parse_args(sys.argv)" | |
| 19:03:41 | sean-k-mooney | cz3: if you do that i think it should work | |
| 19:04:59 | sean-k-mooney | although im not sure you need to initalise rpc or configre the db | |
| 19:05:21 | cz3 | in this case I just need the db access | |
| 19:06:49 | sean-k-mooney | which looks like its jsut sqlalchemy_api.configure(CONF) | |
| 19:07:26 | cz3 | sean-k-mooney: config.parse_args did the trick | |
| 19:07:31 | cz3 | thank you! | |
| 19:07:31 | sean-k-mooney | in any case i would use https://github.com/openstack/nova/blob/master/nova/cmd/manage.py as a reference | |
| 19:07:43 | sean-k-mooney | no worries | |
| 19:08:03 | sean-k-mooney | CONF(argv[1:], | |
| 19:08:05 | sean-k-mooney | project='nova', | |
| 19:08:07 | sean-k-mooney | version=version.version_string(), | |
| 19:08:09 | sean-k-mooney | default_config_files=default_config_files) | |
| 19:08:11 | sean-k-mooney | that is likely the bit you really needed | |
| 19:08:37 | sean-k-mooney | that is invokeing the gloabl constuctor for the singelton | |
| 19:09:11 | cz3 | I actually aim to add something to nova-manage and send it over for review, just wanted to test few things beforehand | |
| 19:09:35 | cz3 | I emailed the openstack-discuss list last week about this, it was re: request_specs regeneration | |
| 19:09:56 | cz3 | where I wanted to find a way to sync extra_specs from flavor data into existing instances | |
| 19:10:33 | sean-k-mooney | oh yes i remember | |
| 19:10:39 | cz3 | long time ago I did something like this by directly appending stuff into request_spec json in database, but this time I want to do it the proper way | |
| 19:13:20 | cz3 | I know that flavors are supposed to be immutable etc. but in this case I just want the existing request_specs to reflect the actual reality and not make the scheduler migrate instances where it was not supposed to after underlying flavor was updated with extra_specs | |
| 19:15:05 | cz3 | I will try to add that to nova-manage and send it over to review via gerrit, maybe it will be good enough to get upstreamed | |
| 19:15:54 | sean-k-mooney | the main consurn we woudl have it that such an update can lead to the vm being broken or on an invalid host also | |
| 19:16:30 | sean-k-mooney | on one hand nova manage is admin only and require direct acess to the server hosting the cloud to get the configs | |
| 19:16:59 | sean-k-mooney | but from a downstream product point of view im not sure we would support customer executing this command | |
| 19:17:16 | sean-k-mooney | we may be with the caveat tha that if you break things you fix them | |
| 19:17:33 | sean-k-mooney | its something we would have to think about | |
| 19:18:01 | cz3 | yeah, I do share the concern, I don't want to send something that would end up breaking deployments for other people if used improperly | |
| 19:18:35 | sean-k-mooney | you proably would need to run som of the flavor/image compatiabity checks before doing the update to make it safe | |
| 19:18:53 | sean-k-mooney | our add a --yes-i-really-really-mean-it flag | |
| 19:19:14 | sean-k-mooney | anyway o/ time to go have dinner | |
| 19:19:42 | cz3 | sure | |
| 19:20:05 | cz3 | thanks again! | |
| 21:57:38 | opendevreview | melanie witt proposed openstack/nova stable/ussuri: zuul: Start to migrate nova-live-migration to zuulv3 https://review.opendev.org/c/openstack/nova/+/795432 | |
| 21:57:39 | opendevreview | melanie witt proposed openstack/nova stable/ussuri: Remove broken legacy zuul jobs https://review.opendev.org/c/openstack/nova/+/795374 | |
| 22:07:55 | tosky | melwitt: an supersquash! Would it make sense to change the first line of the commit message? It is more than just that single commit it was taken from | |
| 22:21:29 | opendevreview | melanie witt proposed openstack/nova master: api: Log correct client IP if load balancer in use https://review.opendev.org/c/openstack/nova/+/786766 | |
| 22:25:02 | gmann | stephenfin: can you check if anything else you want to add in this spec, leaving +w to you https://review.opendev.org/c/openstack/nova-specs/+/791620 | |
| #openstack-nova - 2021-06-10 | |||
| 00:02:37 | opendevreview | melanie witt proposed openstack/nova master: Add functional regression test for bug 1853009 https://review.opendev.org/c/openstack/nova/+/695012 | |
| 00:02:37 | opendevmeet | bug 1853009 in OpenStack Compute (nova) ussuri "Ironic node rebalance race can lead to missing compute nodes in DB" [High,In progress] https://launchpad.net/bugs/1853009 - Assigned to Mark Goddard (mgoddard) | |
| 00:02:37 | opendevreview | melanie witt proposed openstack/nova master: Clear rebalanced compute nodes from resource tracker https://review.opendev.org/c/openstack/nova/+/695187 | |
| 00:02:37 | opendevreview | melanie witt proposed openstack/nova master: Invalidate provider tree when compute node disappears https://review.opendev.org/c/openstack/nova/+/695188 | |
| 00:02:37 | opendevreview | melanie witt proposed openstack/nova master: Prevent deletion of a compute node belonging to another host https://review.opendev.org/c/openstack/nova/+/694802 | |
| 00:02:38 | opendevreview | melanie witt proposed openstack/nova master: Fix inactive session error in compute node creation https://review.opendev.org/c/openstack/nova/+/695189 | |
| 06:39:39 | masterpe[m] | I have more resources declined in the placement.allocations table then actual in use. I see some instance ID there that are deleted. | |
| 06:39:56 | masterpe[m] | What shall I do with them? | |
| 06:46:16 | frickler | masterpe[m]: seems this tool was made for you https://docs.openstack.org/nova/latest/cli/nova-manage.html#placement-audit | |
| 06:46:53 | frickler | or maybe the heal-allocations above | |
| 06:52:57 | masterpe[m] | nice thanks | |
| 08:06:14 | lyarwood | kashyap: https://review.opendev.org/c/openstack/nova/+/795533/5#message-3e7b1dc4d3f8d22b2c9a55637c7366199d00eb56 - I'll write this up in a Nova bug and likely libvirt bug later today but would you mind scanning this if you get a chance? | |
| 08:06:37 | kashyap | lyarwood: Mornin; /me clicks | |
| 08:07:30 | kashyap | lyarwood: Ah, you're debugging the informative error, "reason=failed" | |
| 08:07:45 | lyarwood | there's two parts really | |
| 08:07:51 | lyarwood | yeah the reason=failed thing | |
| 08:08:18 | lyarwood | and the way that the call to virDomainMigrateToURI3 from the source doesn't pick up the failure on the dest | |
| 08:09:23 | kashyap | Hmm, yikes | |
| 08:09:25 | lyarwood | I'm assuming that the dest tears down the connection between the two leading to the eventual `unable to connect to server` error | |
| 08:10:09 | lyarwood | instead of the source being told the migration has failed etc | |
| 08:10:47 | kashyap | lyarwood: This is the dest, right: https://zuul.opendev.org/t/openstack/build/f3b829801901417c9310ad5cc5a0e886/log/controller/logs/libvirt/libvirtd_log.txt | |
| 08:10:54 | lyarwood | yeah | |
| 08:11:15 | kashyap | Is it just me or is it loading deadly slow? I just want to pull down the entire raw file | |
| 08:11:50 | lyarwood | yeah these files are pretty large, I typically pull them down now | |
| 08:12:02 | kashyap | My browser is hung here (probably it goes into MBs). Do I clicck on the "View log" to get the raw link? | |
| 08:12:15 | lyarwood | it's awkward as the raw links don't use .gz at the end of the file names | |
| 08:12:17 | kashyap | Maybe I can simply `wget` the above URL - /me tries | |
| 08:12:37 | lyarwood | mv it to a .gz file and either gunzip or let vim unpack them btw | |
| 08:12:40 | kashyap | Right, but it _is_ a .gz file - a new user will discover it after your browser crashes :D | |
| 08:12:53 | kashyap | lyarwood: Yeah; that's become "muscle memory" now | |
| 08:12:54 | lyarwood | browsers can handle .gz | |
| 08:14:08 | kashyap | Oh, sure; sometimes, a very large file just crashed FF for me | |
| 08:15:31 | kashyap | Actual link to `wget` is: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_f3b/795533/5/check/nova-next/f3b8298/controller/logs/libvirt/libvirtd_log.txt | |
| 08:15:36 | kashyap | 93MB only | |
| 08:32:27 | kashyap | lyarwood: Two quick things: is this reproducible? Or is it the first you noticed? | |
| 08:32:42 | lyarwood | It's the first time I've seen this | |
| 08:32:52 | kashyap | lyarwood: Also, strangely, I don't see the migrateToURI3() failure in the source libvirtd log: | |
| 08:32:57 | kashyap | I pulled the 113MB file from here: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_f3b/795533/5/check/nova-next/f3b8298/compute1/logs/libvirt/libvirtd_log.txt | |
| 08:33:12 | kashyap | I wonder if it got log-rotated | |
| 08:34:39 | opendevreview | Wenping Song proposed openstack/nova master: Replaces tenant_id with project_id from List/Update Servers APIs https://review.opendev.org/c/openstack/nova/+/764292 | |
| 08:34:39 | opendevreview | Wenping Song proposed openstack/nova master: Replace all_tenants with all_projects in List Server APIs https://review.opendev.org/c/openstack/nova/+/765311 | |