| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-08-02 | |||
| 08:20:22 | bauzas | I just saw melwitt's thread about the CI issue in this weekend | |
| 08:20:43 | bauzas | but we didn't have any problems before | |
| 08:23:05 | gibi | bauzas: yepp that is time sensitive, it hits on every last day of a month | |
| 08:23:19 | gibi | so it only failed on 31th | |
| 08:23:45 | gibi | melwitt: has a fix proposed but I'd like to test is in a VM where the clock is set back to 31th | |
| 08:23:55 | gibi | s/test is/ test it/ | |
| 10:52:29 | lyarwood | gibi / sean-k-mooney / stephenfin ; https://github.com/openstack/nova/blob/5a56ac42ad24d74a2bc150769fd2d2fdcbfa3fd6/nova/compute/manager.py#L8873-L8875 - stupid question, is this actually a no-op for Neutron? If so any objections if I remove it? | |
| 10:53:12 | lyarwood | unless it's already called | |
| 10:53:44 | lyarwood | oh right it's already called during pre_live_migration | |
| 10:58:27 | opendevreview | Lee Yarwood proposed openstack/nova master: WIP compute: Remove no-op setup_networks_on_host calls https://review.opendev.org/c/openstack/nova/+/803191 | |
| 11:20:32 | gibi | lyarwood: yepp | |
| 11:25:23 | lyarwood | kk I'll see what ^ borks in terms of tests and go from there | |
| 11:25:42 | dbulira | filters or Placement. Is there a way of setting up separate quotas for various vCPUs (cores) for a single project? Similarly what is done in Cinder when you have general quota for gigabytes and per-storage-type specific one. Any hint where to look at is highly welcomed. | |
| 11:25:42 | dbulira | Hi! Sorry for dropping the question here, however didn't get attention in the general channel. I have a general question regarding quotas in Nova. Let's assume I have two tiers of vCPUs lowfreq and highfreq or any other division (AMD/Intel, etc.). They are of course kept in separate host-aggregates and instance-host filtering is done via scheduler | |
| 13:39:10 | opendevreview | Jacek Tomasiak proposed openstack/nova stable/rocky: Reject open redirection in the console proxy https://review.opendev.org/c/openstack/nova/+/803182 | |
| 14:43:13 | opendevreview | Merged openstack/nova master: Restore original time override in test_archive_task_logs https://review.opendev.org/c/openstack/nova/+/803106 | |
| 15:24:59 | gibi | stephenfin: do you mean missing unit test coverage here? https://review.opendev.org/c/openstack/nova/+/793620/13#message-51af15791583c674361ebd329cab0759e2297d4d | |
| 15:25:31 | gibi | stephenfin: as there is a set of functional tests covering the modified APIs there | |
| 16:00:24 | opendevreview | Alexey Stupnikov proposed openstack/nova master: Clean up unreal UUID values in tests https://review.opendev.org/c/openstack/nova/+/803226 | |
| 16:42:50 | opendevreview | Sylvain Bauza proposed openstack/nova master: Provide the mdev class for every PCI device https://review.opendev.org/c/openstack/nova/+/802918 | |
| 16:42:51 | opendevreview | Sylvain Bauza proposed openstack/nova master: Provide and use other RCs for mdevs if needed https://review.opendev.org/c/openstack/nova/+/803233 | |
| 16:42:51 | opendevreview | Sylvain Bauza proposed openstack/nova master: DNM (yet) : Expose the mdev class https://review.opendev.org/c/openstack/nova/+/801743 | |
| 17:15:10 | opendevreview | Sylvain Bauza proposed openstack/nova master: WIP : Expose the mdev class https://review.opendev.org/c/openstack/nova/+/801743 | |
| 17:15:10 | opendevreview | Sylvain Bauza proposed openstack/nova master: Provide and use other RCs for mdevs if needed https://review.opendev.org/c/openstack/nova/+/803233 | |
| 22:19:12 | opendevreview | Samuel proposed openstack/nova-specs master: Migrate Instance Between Projects https://review.opendev.org/c/openstack/nova-specs/+/802034 | |
| #openstack-nova - 2021-08-03 | |||
| 01:03:41 | songwenping | bauzas, sean-k-mooney[m]: thanks for last week's response. | |
| 07:29:25 | bauzas | gibi: morning | |
| 07:29:35 | bauzas | looks like the nova-lvm job is on the weeds | |
| 07:32:15 | gibi | bauzas: o/ I have to be off for 2 hours | |
| 07:33:12 | bauzas | no worries, I'll try to look at the problme | |
| 07:33:12 | gibi | it seems it is in the weeds for daays | |
| 07:33:15 | bauzas | yup | |
| 07:33:28 | bauzas | a snapshot not found or something like this | |
| 07:33:52 | gibi | last success was 2021-07-16T08:36:39 | |
| 07:34:08 | bauzas | yep http://logstash.openstack.org/#dashboard/file/logstash.json?query=build_name%3A%5C%22nova-lvm%5C%22%20build_status%3A%5C%22FAILURE%5C%22 | |
| 07:34:16 | bauzas | (if logstash is still working correctly) | |
| 07:35:25 | gibi | but then how we managed to merge things?! | |
| 07:35:43 | gibi | was this job turned off? | |
| 07:36:43 | gibi | or is it selectively runnig? | |
| 07:37:36 | gibi | it doesn't have a too special irrelevant-files in .zuul.yaml | |
| 07:37:40 | gibi | so I don't get it | |
| 07:37:48 | gibi | I have a bunch of patches that nova-lvm isn't run on | |
| 07:38:00 | bauzas | excellent question | |
| 07:38:11 | gibi | like https://review.opendev.org/c/openstack/nova/+/800086/13 | |
| 07:38:23 | gibi | pingig lyarwood :) | |
| 07:38:27 | bauzas | zuul is pretty https://zuul.openstack.org/builds?job_name=nova-lvm | |
| 07:38:44 | bauzas | none of the job builds worked since 2 weeks | |
| 07:38:44 | gibi | and I'm dropping off now. be back later | |
| 07:38:51 | bauzas | gibi: no worries and thanks | |
| 07:40:05 | yonglihe | HI, everyone | |
| 07:42:55 | bauzas | ok, the nova-lvm job is triggered on libvirt changes hence the non-failure on other changes and why we merged other stuff | |
| 07:43:01 | bauzas | https://zuul.openstack.org/job/nova-lvm | |
| 07:43:39 | bauzas | now, looks like all runs fail with the same tempest tests with the same exception being a snapshot not found | |
| 07:46:33 | bauzas | huzzaaaaah \o/ https://zuul.openstack.org/build/a04b0d2e184a4d3e9c39bd6110b454ea/log/controller/logs/screen-n-cpu.txt?severity=4#12685-12689 | |
| 07:46:44 | bauzas | NotImplementedError: direct_snapshot() is not implemented | |
| 07:47:12 | bauzas | looks like we regressed | |
| 07:51:29 | bauzas | hmmmm | |
| 07:51:53 | lyarwood | Morning | |
| 07:52:09 | lyarwood | yeah I've not had time to look at nova-lvm failures thanks to a few downstream fires | |
| 07:52:17 | lyarwood | they should be out now so I'll look this morning | |
| 07:52:20 | bauzas | ok, I have better understanding | |
| 07:52:22 | lyarwood | bauzas: that isn't the failure | |
| 07:52:25 | bauzas | correct | |
| 07:52:33 | bauzas | we catch the notimplemented exception | |
| 07:52:36 | bauzas | it's failing later | |
| 07:52:51 | bauzas | when calling glance I'd say | |
| 07:53:06 | bauzas | https://zuul.openstack.org/build/a04b0d2e184a4d3e9c39bd6110b454ea/log/controller/logs/screen-n-cpu.txt?severity=4#12772 | |
| 07:53:08 | lyarwood | yeah the g-api 502 is something I think I've seen before when we run out of space during the upload | |
| 07:53:27 | bauzas | yup | |
| 07:53:57 | bauzas | we want to upload the snapshot and we fail, right? | |
| 07:54:22 | lyarwood | we are in the process of uploading the snapshot and something fails | |
| 07:54:29 | lyarwood | it might even be a timeout in httpd | |
| 07:54:39 | lyarwood | given the size of the images we are uploading for these tests | |
| 07:55:18 | lyarwood | brb | |
| 07:56:23 | bauzas | lyarwood: https://zuul.openstack.org/build/a04b0d2e184a4d3e9c39bd6110b454ea/log/controller/logs/screen-g-api.txt?severity=3#3555 | |
| 07:56:37 | bauzas | this error matches the req-id of the n-cpu issue | |
| 08:11:54 | bauzas | well, can't figure out more why the temporary image fails | |
| 08:12:09 | bauzas | abhishekk: around ? | |
| 08:12:43 | bauzas | abhishekk: we have a nova job failure (nova-lvm) which fails constantly because of glance API returning 502 https://zuul.openstack.org/build/a04b0d2e184a4d3e9c39bd6110b454ea/log/controller/logs/screen-g-api.txt?severity=3#3555 | |
| 08:12:55 | bauzas | on an image upload | |
| 08:34:47 | abhishekk | bauzas, looking | |
| 08:35:02 | bauzas | abhishekk: thanks, wrote an email about this | |
| 08:35:10 | bauzas | (to summarize) | |
| 08:35:17 | abhishekk | yes, just went through it | |
| 08:36:35 | abhishekk | bauzas, are you sure failure is because of this warning ? | |
| 08:37:01 | bauzas | abhishekk: I'm nearly sure the test is failing because glance is returning a 502 | |
| 08:37:25 | abhishekk | ok, will have a look and get back to you | |
| 08:38:05 | bauzas | abhishekk: but I relate the glance g-api warning with the same req-id of the failing glance image upload | |
| 08:38:25 | bauzas | correlation isn't consequence, I know, but that's a starting point | |
| 08:38:33 | abhishekk | ack | |
| 08:39:17 | lyarwood | bauzas: the test is 100% failing because of the 502 it just isn't clear why g-api or httpd is returning it | |
| 08:39:27 | lyarwood | sorry got my head stuck downstream again | |
| 08:39:40 | bauzas | yup, that's what I said to abhishekk | |
| 08:39:53 | bauzas | we have an image upload failure | |
| 08:40:00 | bauzas | but we don't know why | |
| 08:40:57 | bauzas | lyarwood: no worries about downstream rush, I feel your pain, maybe just one thought : as this is failing, should we make the job non-voting until we figure out the cause ? | |
| 08:41:16 | lyarwood | Yup happy to if you can raise a bug | |
| 08:41:29 | bauzas | good call, I forgot this :facepalm: | |
| 08:43:06 | abhishekk | I can see HTTP 413 in g-api log | |
| 08:46:20 | bauzas | lyarwood: abhishekk: gibi: https://bugs.launchpad.net/nova/+bug/1938765 | |