Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2022-01-17
15:37:13 sean-k-mooney im pretty sure there is a way to defeint them as a mutualy exclucive group
15:37:30 sean-k-mooney i just cant think of a place off the top of my head whwere we actully do that
15:37:43 sean-k-mooney i suspect this si normally check after the scheme validation
15:39:32 Uggla Is making both options ok in schema validation and then doing the exclusive part in the code, sounds ok ?
15:39:38 sean-k-mooney https://github.com/openstack/nova/blob/1ddb8f83adef964a8ca050994a43adc6175994f1/nova/api/openstack/compute/schemas/servers.py#L172-L173
15:39:45 sean-k-mooney oneOf
15:40:09 Uggla sean-k-mooney, nice thank you !
15:40:42 sean-k-mooney as i said we rearly enforce that in teh scheme but no one will object if you do it either
15:41:40 sean-k-mooney there are some other exampels in the samee file
15:41:41 sean-k-mooney https://github.com/openstack/nova/blob/1ddb8f83adef964a8ca050994a43adc6175994f1/nova/api/openstack/compute/schemas/servers.py#L281-L298
15:42:58 bauzas sean-k-mooney: I'll create the specs directory once we know the Z name :)
15:43:26 bauzas unless I name it 'zombie' directly :p
15:45:24 sean-k-mooney oh ya good point
15:46:29 sean-k-mooney i still quite like "zenith" cause its all down hill from there
15:53:16 gmann bauzas: gibi sean-k-mooney Uggla artom yeah, we can remove the AZ from being 'required' from schema now as it will ne extended to accepting 'host' too. AZ was added as 'required' because this API only accept None or AZ and no other field but that is handled by additionalProperties=False so 'required' is not needed as such.
15:54:09 gmann bauzas: I think you will be right about 'zombie' :) everyone liking it :)
15:56:36 sean-k-mooney if its is zombie we shoudl make it a base for a downstream realse
15:57:01 sean-k-mooney because a decade form now we will still have once custoemr that refuses to move off it and it will never die
15:57:10 sean-k-mooney like any good zombie :)
15:58:02 gmann :)
15:59:52 sean-k-mooney osp 18 actuly should either be the z release of a release depending on timeing so maybe that will happen anyway
16:00:12 sean-k-mooney *Z release or A release
16:00:59 bauzas A = Awesome release I guess
16:01:17 bauzas easy peasy
16:01:28 sean-k-mooney B = Buffy to help with the zombie problem ?
16:02:01 bauzas C = Chilly
16:02:14 bauzas D = Draughful
16:03:51 Uggla I like B especially if Z is zombie !
16:14:36 Uggla Sarah Michelle Gellar was so cute.
16:15:55 opendevreview Merged openstack/nova master: ensure samples folder exists for microversion https://review.opendev.org/c/openstack/nova/+/813672
17:39:16 opendevreview Merged openstack/nova master: libvirt: Ensure all volume drivers log the instance whenever possible https://review.opendev.org/c/openstack/nova/+/780260
18:17:22 opendevreview Merged openstack/nova master: functional: Add reproducer for #1907775 https://review.opendev.org/c/openstack/nova/+/766771
19:44:05 ade_lee sean-k-mooney, lyarwood hey guys, I updated the fips job to include the full set of tempest tests and there were quite a few more failures -- can you guys take a look?
19:44:12 ade_lee https://review.opendev.org/c/openstack/nova/+/790519
19:45:44 ade_lee cinder folks are looking into other potential problems with volumes / cryptsetup here -- https://review.opendev.org/c/openstack/cinder/+/790535 so some of the failures may be related to that.
19:46:30 sean-k-mooney sure
19:47:05 sean-k-mooney have the general centos8 gate issue been fixed
19:47:17 sean-k-mooney i.e. the fact they broke ping
19:47:23 sean-k-mooney making it require root to run
19:47:55 ade_lee sean-k-mooney, not sure .. that woud certainly muddy things
19:48:14 ade_lee fungi, clarkb ^^ any idea?
19:49:01 fungi sean-k-mooney: ade_lee: i don't know if red hat has fixed centos stream 8's packages yet
19:49:42 sean-k-mooney the fips failrues might be unrealtaed but that is breaking all centos8 stream jobs currently right
19:49:45 fungi technically, centos stream 8 was (maybe still is) broken/regressed. i recommend not relying on it since it doesn't seem like they actually test it like they did the old centos
19:50:18 sean-k-mooney fungi: for what its worth i agree with not working aroudn this in our jobs
19:50:21 fungi i think there's work underway to get rocky linux added, and we have openeuler though it sounds like it may have a slightly too old kernel for the fips testing
19:50:50 sean-k-mooney fungi: really centos 8 stream should be fixed and maintianed as if it was rhel
19:51:36 sean-k-mooney ade_lee: qemu-kvm: -accel tcg: mprotect of jit buffer: Cannot allocate memory
19:51:36 fungi rocky or alma seems like it might be a better option if centos stream is going to be the unstable hopper where free users get to find the bugs and tell red hat what to fix before they tag a new release of their commercial product
19:52:41 sean-k-mooney fungi: as someone who has to maintain said product if they dont keep centos stream stable i will continue to do all my dev on ubuntu
19:54:23 sean-k-mooney i really dont like how closely tied our openstack product is to rhel or the rhel lifecycle today. if it is to contibute more value then the bruden it brings they need to keep centos 8 stream stable or make licenses avaiabel for opensrouce to sue for ci on rhel itself
19:54:50 fungi but yeah, maybe this is a one-off, or maybe it'll be the kick needed to get more thorough testing in place for centos stream package updates
19:54:51 sean-k-mooney if they dont using rocky or alma and not support centos makes complete sense
19:55:53 sean-k-mooney fungi: we have a log runnign downstream backlog itme to get a centos stream devstack jobs running in our donwstream ci at somepoint but making it work with our patched repos is non tivial
19:56:11 ade_lee sean-k-mooney, so -- the mprotect thing -- what does that mean?
19:56:33 sean-k-mooney ade_lee: it looks like the host vm ran out of memory
19:57:46 ade_lee sean-k-mooney, so we should be testing using larger vms? I think I recall folks having to do that somewhere -- checking ..
19:58:05 sean-k-mooney ade_lee: no you need to reduce the concurancy
19:58:12 sean-k-mooney ade_lee: how have you defiend the regex
19:58:20 fungi unless you can get by with smaller flavors for the cirros guests or something
19:58:24 sean-k-mooney the senario test need to be run serially after the other test
19:59:03 sean-k-mooney fungi: the vms are using 128mb flavors
19:59:12 sean-k-mooney that should be ok
19:59:41 sean-k-mooney i wonder if the host vm is missing the extended swap
20:00:05 sean-k-mooney that we are using to workaround the 1G tcg buffer
20:00:13 ade_lee sean-k-mooney, the cinder tests are running with swap .. https://review.opendev.org/c/openstack/cinder/+/790535/24/.zuul.yaml
20:00:22 ade_lee configure_swap_size: 4096
20:00:39 sean-k-mooney that proably needs to be increased
20:01:00 sean-k-mooney fungi: do you know what we set it to for the debian jobs
20:01:31 ade_lee sean-k-mooney, ack - well , its not set at all in the nova tests we just ran -- so we need at least that ..
20:01:50 sean-k-mooney devstack allocates swap by default i belive
20:02:17 sean-k-mooney 4096 for centos https://github.com/openstack/devstack/blob/3c98c21fec60da8d2d39df2e1d9b845a51817a0e/.zuul.yaml#L644
20:03:16 sean-k-mooney which is the same as debain https://github.com/openstack/devstack/commit/f8e00b86aee9a8f9646bf5aed2c618843307b963
20:04:21 sean-k-mooney im not sure if the vars section are merged
20:04:25 sean-k-mooney https://review.opendev.org/c/openstack/nova/+/790519/18/.zuul.yaml
20:04:32 sean-k-mooney so you might be overwriting it
20:05:19 ade_lee sean-k-mooney, we made the same change in swift tests - and got better results ..
20:05:23 sean-k-mooney ya you are https://0d2f8596f517931fcac8-e8ce9722d0830f321adb7d7d98f7ea38.ssl.cf1.rackcdn.com/790519/18/check/nova-centos8-stream-fips/372ef4e/zuul-info/inventory.yaml
20:05:45 sean-k-mooney ade_lee: swift does not spwan vms
20:06:06 ade_lee sean-k-mooney, ack
20:06:14 sean-k-mooney the swap is needed because qemu started allocating a 1GB cache for tcg byty code per qemu process
20:06:41 ade_lee sean-k-mooney, so try 8192?
20:06:44 sean-k-mooney meanign your 128mb vms now takes 1156mb
20:06:53 sean-k-mooney well right now you have 0
20:07:03 sean-k-mooney so add configure_swap_size: 4096
20:07:12 ade_lee ok so lets try 4096 ..
20:07:33 sean-k-mooney you can bump it to configure_swap_size: 8192 but we also only have 80G of disk space in the vms so dont set it too large
20:07:48 sean-k-mooney fungi: have we considerd using zram by the way
20:07:58 sean-k-mooney in the long run that might help in general
20:08:53 opendevreview Ade Lee proposed openstack/nova master: Add check job for FIPS https://review.opendev.org/c/openstack/nova/+/790519
20:09:32 sean-k-mooney lets see if that helps and we can take another look tomorow
20:15:08 ade_lee sean-k-mooney, ack
20:27:04 fungi sean-k-mooney: i'm not familiar with zram, is that on-the-fly memory compression?
20:53:17 sean-k-mooney fungi: yes basically it creats a compressed ramdisk that you then confirure for swap with a higher priority then normal swap on disk
20:53:39 sean-k-mooney with zstd compression you can get up to a 4x ratio
20:53:59 sean-k-mooney fungi: fedora started using it by default for laptops and low memeory isntalls
20:54:32 fungi mmm... then why not just compress the ram itself? i guess it's a simple composition to do the same, the extra layers just give me a moment's pause
20:55:03 sean-k-mooney fungi: it is simpler to integrate in the kernel this way
20:55:34 sean-k-mooney you can use zswap which will do it transparently but zram+swap has is more compatiable
20:55:55 sean-k-mooney windows and osx just compress ram transparently

Earlier   Later