Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2023-04-14
16:46:50 noonedeadpunk At we both have dhcp enabled
16:47:27 sean-k-mooney ya so the irccloud linke has ip_address in the data
16:47:40 sean-k-mooney which is what i expect in the networks section
16:48:20 sean-k-mooney that has "type": "ipv4"
16:48:31 sean-k-mooney vs "type": "ipv4_dhcp"
16:49:04 noonedeadpunk but it's based on `subnet.get_meta('dhcp_server')`?
16:49:10 sean-k-mooney its been a while but if i recall correctly when its type ipv4_dhcp that is not ment to have teh ip info where as tyep ipv4 is for static netwrokgin without dhcpu
16:49:35 noonedeadpunk specifically this block I assume https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L292-L297
16:49:43 sean-k-mooney if the subnet does not have dhcpu enabled we store the static ip info so cloud init can configure it
16:49:55 sean-k-mooney but if it has dhcpu enabel we expect cloud init to use dhcpu
16:50:23 noonedeadpunk ok, yes, I see. I wonder how we managed to get this working for years then...
16:50:36 noonedeadpunk that is totally different question though
16:50:41 noonedeadpunk thanks sean-k-mooney!
16:50:49 sean-k-mooney noonedeadpunk: perhaps you modifed the template
16:50:58 noonedeadpunk have a good weekend
16:51:05 johnsom That is odd given I have DHCP on the subnet, but the metadata still has the fixed IP
16:51:19 sean-k-mooney https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.injected_network_template
16:51:40 noonedeadpunk I for sure have not changed that
16:52:02 sean-k-mooney thats good becasue that is not tested anywhwere
16:52:05 noonedeadpunk I was debugging OSA CI job that start failing month ago
16:52:06 sean-k-mooney and has not been for years
16:52:12 sean-k-mooney so it proably does not work
16:52:21 noonedeadpunk Or well. I t was passing last time month ago
16:52:58 noonedeadpunk and in the meanwhile what we changed was nova/octavia/neutron versions we're isntalling and ansible collection that creates networks
16:53:12 sean-k-mooney so thats this https://github.com/openstack/nova/blob/master/nova/virt/interfaces.template but thats not actuly for the data your looking at
16:53:25 noonedeadpunk but it's interesting why me and johnsomhave different result given we both have ddhcp enabled
16:53:54 sean-k-mooney johnsom: noonedeadpunk what ml2 drivers are you using
16:54:02 sean-k-mooney ml2/ovs ml2/ovn
16:54:21 sean-k-mooney this code should not care but just wondering
16:55:13 noonedeadpunk ovn
16:55:18 johnsom ovn
16:55:37 noonedeadpunk btw, upgrade job that is passing is using lxb
16:55:56 noonedeadpunk (upgrade from Y to 2023.1)
16:56:00 sean-k-mooney ok so it started breaking in ovn but only somethimes
16:56:59 noonedeadpunk I have a sandbox vm where it's reproducible
16:57:04 johnsom We haven't seen any changes in the Octavia jobs, they are all still passing as expected.
16:57:15 sean-k-mooney johnsom: noonedeadpunk are ye seeign this in yoru downstream installs or is this also seen in upstram devstack tempest jobs
16:57:42 noonedeadpunk but it's in OSA jobs
16:57:42 sean-k-mooney noonedeadpunk: your using OSA right
16:57:49 noonedeadpunk yup
16:57:56 noonedeadpunk No idea about downstream yet
16:58:05 sean-k-mooney ya im wondering if there is some neturon/nova config setting at play that changes this behvior
16:58:12 sean-k-mooney or an enabled neutron api extention
16:58:24 noonedeadpunk well, it could be...
16:58:29 sean-k-mooney most of that code has not changed in 4-6 years
16:58:40 noonedeadpunk we changed neither of that and yeah - code is quite old indeed
16:58:46 sean-k-mooney but that does not mean other code around it had not changed the data its reciving
16:59:05 noonedeadpunk I wonder if that could boil down to smth like sdk version :D
16:59:29 sean-k-mooney not in this case since we are not using the sdk altough maybe neutron clinet
16:59:31 noonedeadpunk that changes output of `subnet.get_meta('dhcp_server')`
16:59:54 noonedeadpunk but that should be neutron client object?
17:00:43 noonedeadpunk sorry really need to run as otherwise my wife will feed my cold corps to pigs
17:00:44 sean-k-mooney not nessisarly
17:00:53 sean-k-mooney cool
17:00:59 sean-k-mooney ping us on monday
17:01:13 noonedeadpunk sure, sorry for that :)
17:01:19 noonedeadpunk have a good weekends again
17:01:56 sean-k-mooney no its an iteresting issue
17:02:15 sean-k-mooney i was jut checkign if it was perhaps oru Subnet model object we use form the network info case but no
17:02:17 sean-k-mooney https://github.com/openstack/nova/blob/master/nova/network/model.py#L299-L339
17:02:28 sean-k-mooney that does not have .get_meta
17:06:38 sean-k-mooney https://github.com/openstack/nova/commit/c7f572d65b57e034c1391ed49d84f6e5f1d672ad seams related
17:07:15 sean-k-mooney johnsom: noonedeadpunk when ye are aroudn on monday check if there is a dhcp port on the network in question
17:07:55 sean-k-mooney this might be caused by useing ovn with or without the dhcp agent
17:08:03 sean-k-mooney if deployed without it we might not have the info
17:08:10 sean-k-mooney btu with it we would?
17:08:15 sean-k-mooney that is my best guess right now
17:08:25 dansmith bauzas: you would not believe how many unit tests depend on this broken behavior (incorrectly)
17:09:42 sean-k-mooney johnsom: looking at https://github.com/openstack/nova/blob/master/nova/virt/netutils.py#L292-L297
17:09:55 sean-k-mooney we dont have the same workaroudn for that edgecase
17:10:36 johnsom I don't have the dhcp agent running, mine is pure ovn
17:10:55 sean-k-mooney and you have the info
17:11:02 johnsom correct
17:11:09 sean-k-mooney ok lets see if the its the same for noonedeadpunk on monday
17:11:43 sean-k-mooney the way we are doing the check might still be realted but im not sure
17:15:46 sean-k-mooney johnsom: this si workinf form info in the network infocache by the way
17:16:23 sean-k-mooney https://github.com/openstack/nova/blob/ae42400b7663bc58d5562de99e976c95131b77a9/nova/virt/netutils.py#L168-L224
17:17:35 sean-k-mooney ovn is racy with how its sends network_vif_plugged events
17:17:48 sean-k-mooney so its posible there is a race here
17:19:02 dansmith ugh, the whole point of network_vif_plugged was to close a race :/
17:20:26 sean-k-mooney ya i know but ovn did not have a way orgian to knwo if the flwo were programed
17:20:39 sean-k-mooney so it said it was pluged when it was bound
17:20:50 dansmith yeah, that's just unfortunate
17:20:53 sean-k-mooney now it has a websocket callback meachnium
17:21:08 sean-k-mooney but it still does not wait for the ovn metadta agaent to compltes
17:22:04 sean-k-mooney which makes metadata and i think dncp racy on boot which as you know is why network vif plugged was added
17:22:27 sean-k-mooney we dont have the dhcp race if using ovn native dhcp
17:22:49 sean-k-mooney but if you use the dhcp agent for ironic support i think it has a race but not sure
19:09:47 noonedeadpunk I'd say my attempts fail quite reliably many times in a row without passing any single time
19:10:15 noonedeadpunk so doesn't look like race in my case at least
19:26:10 noonedeadpunk thankfully it's super easy to fix now. But this doesn't really solve the mystery of why we're having different results with same input
20:38:24 opendevreview melanie witt proposed openstack/nova master: Call volume detach rollback API if detach fails https://review.opendev.org/c/openstack/nova/+/880399
21:30:19 opendevreview Merged openstack/nova master: Update to the PTL guide https://review.opendev.org/c/openstack/nova/+/875730
22:36:52 opendevreview Robert Breker proposed openstack/nova master: Fix nova-manage image_property show unexpected keyword https://review.opendev.org/c/openstack/nova/+/880557
#openstack-nova - 2023-04-17
01:08:27 opendevreview Merged openstack/nova master: db: Remove legacy migrations https://review.opendev.org/c/openstack/nova/+/872428
09:05:30 Uggla sean-k-mooney, gibi, bauzas I'm facing an issue. I can mount a share on the compute if I'am an admin, but not a regular user. I think it is something around privsep but I did not figure what it could be. Any idea what could be the issue ?
09:06:57 gibi Uggla: how this fails?
09:07:06 Uggla I receive this error as a regular user : mount.nfs: mounting 192.168.122.76:/opt/stack/data/manila/mnt/share-7643c9cb-bf52-4263-b048-2b3589451cd9 failed, reason given by server: No such file or directory\n'
09:07:24 Uggla however the directory exists.
09:08:54 Uggla as an example using the mount -t nfs 192.168.122.76:/opt/stack/data/manila/mnt/share-7643c9cb-bf52-4263-b048-2b3589451cd9 /opt/stack/data/nova/mnt/57c1c5f49444e0ccdcd1d46b5d640827 command as root is ok.

Earlier   Later