| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-09-16 | |||
| 10:39:00 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove 'NovaObjectDictCompat' from 'InstancePCIRequest' https://review.opendev.org/723573 | |
| 11:47:17 | nightmare_unreal | what can be the cause of tempest failures ? is it related to user patch or infrastructure/zuul issue ? | |
| 11:50:21 | sean-k-mooney | nightmare_unreal: both | |
| 11:50:35 | sean-k-mooney | normlly user patches | |
| 11:50:44 | sean-k-mooney | the jobs can fail for infra reasons | |
| 11:50:53 | sean-k-mooney | but then that normlaly happens before tempest runs | |
| 11:50:59 | gibi | stephenfin: one small question / request in https://review.opendev.org/#/c/745907/9/nova/tests/functional/regressions/test_bug_1675570.py@131 | |
| 11:51:18 | sean-k-mooney | in rare ocation tempets test will fail because of infra if for example the iops of the disk are too low | |
| 11:51:57 | sean-k-mooney | nightmare_unreal: but normally tempest failures are because fo the use patch. unless its a POST_FAILURE which si almost always infra related | |
| 11:52:04 | nightmare_unreal | sean-k-mooney: in my case osc-tx-py36-tips, tempest-full-py3, osc-functional-devstack and osc-functional tips failed | |
| 11:52:42 | sean-k-mooney | nightmare_unreal: which patch? | |
| 11:52:49 | nightmare_unreal | https://review.opendev.org/#/c/742210/ | |
| 11:52:54 | nightmare_unreal | status is post_failure | |
| 11:53:03 | nightmare_unreal | not for tempest | |
| 11:53:04 | sean-k-mooney | ya so post_failure | |
| 11:53:07 | sean-k-mooney | is not your problem | |
| 11:53:21 | sean-k-mooney | that means it failed in one of the post jobs like uploading the logs | |
| 11:53:35 | sean-k-mooney | that is an infra issue and you can just recheck | |
| 11:53:38 | nightmare_unreal | Oh, thanks . will recheck help | |
| 11:53:42 | nightmare_unreal | ah okay | |
| 11:53:44 | sean-k-mooney | the tempst run is only 16m long | |
| 11:53:47 | gibi | stephenfin: I'm totally OK to do an hard coded uuid GlanceFixture.imageX reference replacemnet in a separate patch | |
| 11:53:47 | nightmare_unreal | thanks :) | |
| 11:53:48 | sean-k-mooney | let me check that | |
| 11:53:55 | sean-k-mooney | but that looks like it failed to deploy | |
| 11:54:30 | sean-k-mooney | ERROR: No matching distribution found for oslo.service===2.4.0 (from -c /opt/stack/requirements/upper-constraints.txt (line 306)) | |
| 11:54:47 | sean-k-mooney | nightmare_unreal: so the tempest job failed because it could not install oslo.service | |
| 11:54:56 | sean-k-mooney | that is likely because our mirrors where out of sync | |
| 11:55:03 | nightmare_unreal | oh got it | |
| 11:55:08 | sean-k-mooney | which is also an infa issue | |
| 11:55:24 | nightmare_unreal | I see | |
| 11:55:29 | sean-k-mooney | so i think we can recheck this but good to ask infra if the issue is fixed first | |
| 11:55:35 | nightmare_unreal | I guess I will wait for some time | |
| 11:55:37 | nightmare_unreal | and then recheck | |
| 11:55:46 | nightmare_unreal | okay | |
| 11:56:00 | sean-k-mooney | the zuul run was yesterday | |
| 11:56:04 | nightmare_unreal | i will ask in that channel | |
| 11:56:24 | sean-k-mooney | so i think it shoudl be ok gibi do you know if the mirror issue we hit yesterday is fixed | |
| 11:56:51 | frickler | nightmare_unreal: sean-k-mooney: that was an issue with pypi and should be fixed since around midnight | |
| 11:57:12 | nightmare_unreal | alright :) . I will recheck in that case | |
| 11:57:15 | nightmare_unreal | thanks everyone | |
| 11:57:25 | frickler | we only proxy and cache requests to pypi, no mirroring | |
| 11:57:44 | sean-k-mooney | frickler: ah the mirros are for the distro packages only | |
| 11:58:03 | sean-k-mooney | i tought we have devpi running on the afs file system too | |
| 11:58:54 | frickler | sean-k-mooney: yes, we stopped running bandersnatch when hit the 2T limit a long time ago. yesterday we learning that pypi's own mirror filled up 12T and got stuck with that a month ago | |
| 11:59:22 | sean-k-mooney | ah i see | |
| 11:59:45 | sean-k-mooney | 12TB im guessing it does not prune old verions regularly | |
| 12:00:39 | frickler | well for a full mirror they likely need to keep all versions | |
| 12:00:40 | sean-k-mooney | that or there is a lot of software reguarlly asking for old verions of stuff | |
| 12:00:50 | sean-k-mooney | ah right | |
| 12:01:06 | sean-k-mooney | ya i like devpi because it was a caching proxy for pypi | |
| 12:01:22 | sean-k-mooney | as well as a local pip comparitble data store for your own packages | |
| 12:01:34 | sean-k-mooney | frickler: are we just using squid or something now as a http cache | |
| 12:02:40 | sean-k-mooney | i havent redeployed my onw cache since i moved house and got gigabit but its still vaguly on my todo list after i redeploy my testing third party ci. | |
| 12:05:24 | frickler | we use apache2 because it does both the proxying and serve our mirrored repos at the same time and on the same host, just with different paths | |
| 12:10:05 | sean-k-mooney | ah i might have to look at how that is configured and maybe steal it for my own use at somepoint. having all of infras confic public as a referce is quite helpful that way | |
| 13:06:37 | bauzas | gibi: sorry missed your ping | |
| 13:06:40 | bauzas | you surely can | |
| 13:06:58 | bauzas | gibi: but I'll do some scrubbing too, so lemme know which ones you're looking | |
| 13:16:41 | gibi | bauzas: looking at https://bugs.launchpad.net/nova/+bug/1881944 right now and I think that will be my last for today | |
| 13:16:41 | openstack | Launchpad bug 1881944 in OpenStack Compute (nova) "nova-api returns empty block-device-mapping in metadata queries" [Undecided,New] | |
| 13:16:52 | gibi | nice work on the open bugs, thanks | |
| 13:16:57 | bauzas | gibi: okay, then going on the next ones | |
| 13:18:37 | stephenfin | gibi, bauzas: Thoughts on this bug? https://bugs.launchpad.net/nova/+bug/1741810 tl;dr: since we introduced the ImageMetaProps o.vo way back in Liberty, one can no longer filter on non-standard image metadata properties using 'AggregateImagePropertiesIsolation' | |
| 13:18:37 | openstack | Launchpad bug 1741810 in OpenStack Compute (nova) "Filter AggregateImagePropertiesIsolation doesn't Work" [Undecided,New] | |
| 13:19:11 | stephenfin | so one can filter on 'os_distro=windows' but not 'os=windows' | |
| 13:20:00 | bauzas | stephenfin: looking | |
| 13:20:07 | sean-k-mooney | stephenfin: this sound invaild | |
| 13:20:16 | stephenfin | gibi, bauzas: I have ideas on how to fix it but I wonder if we should? | |
| 13:20:18 | sean-k-mooney | stephenfin: there are nolonger any non-standard image proeprties | |
| 13:20:32 | bauzas | what sean-k-mooney said | |
| 13:20:45 | openstack | Launchpad bug 1893904 in OpenStack Compute (nova) "Placement is not updated if a VGPU is re-created on a new GPU upon host reboot" [Undecided,New] | |
| 13:21:14 | stephenfin | Okay, that's what I'm thinking too. However, glance doesn't place the same restrictions on image metadata properties | |
| 13:21:21 | sean-k-mooney | correct | |
| 13:21:31 | sean-k-mooney | but nova will drop all nonstandard image metadata | |
| 13:21:39 | sean-k-mooney | well no | |
| 13:21:44 | sean-k-mooney | there is a config option for glance | |
| 13:21:50 | stephenfin | So do we just document things, or should we store stuff that's namespaced? | |
| 13:21:51 | sean-k-mooney | that uses the metadefs for validation | |
| 13:22:12 | sean-k-mooney | stephenfin: there are no namespaces in glance metadata | |
| 13:22:22 | sean-k-mooney | at least there was not untill we added trits | |
| 13:22:34 | stephenfin | right, there are traits now | |
| 13:22:38 | stephenfin | i.e. we store anything prefixed with 'aggregate_metadata_' | |
| 13:22:42 | sean-k-mooney | they decied to use prefixes not namespaces | |
| 13:22:54 | stephenfin | namespace, prefix, whatever | |
| 13:23:08 | stephenfin | but just '<namespace>key=value' | |
| 13:23:11 | sean-k-mooney | they are different technically | |
| 13:23:31 | stephenfin | sure, but let's gloss over that for now | |
| 13:23:42 | sean-k-mooney | but ya am i think we shoudl jsut close this as not a bug | |
| 13:23:49 | sean-k-mooney | well invalid | |
| 13:23:58 | stephenfin | wontfix | |
| 13:24:08 | stephenfin | it was a behavior change | |
| 13:24:11 | sean-k-mooney | ah we have that then ya | |
| 13:24:20 | gibi | stephenfin: I agree to close it with some notes about not supporting non-standard props | |
| 13:24:25 | sean-k-mooney | yes but it was not supproted before | |
| 13:24:29 | sean-k-mooney | it just happened to work | |
| 13:24:52 | stephenfin | Okay, I'll do that so. I have a functional test to prove it too. I assume I can just drop that? | |
| 13:25:31 | sean-k-mooney | you could keep it as a negitve regression | |
| 13:25:45 | sean-k-mooney | to ensure we dont acidentaly roll back and enable it | |
| 13:27:14 | gibi | stephenfin: drop it | |