Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2021-11-15
18:00:10 Zer0Byte :s
18:34:21 sean-k-mooney hyang[m]: the addtional request is proably workabole but really since this is a new exttion being added this cycle i woudl jsut change it now before addign the nova support
18:34:45 sean-k-mooney hyang[m]:without another exteion we wont be abel to detech support for usign a signel query
18:48:18 sean-k-mooney hyang[m]: im going to call it a day. tl;dr if this api had already shipped in an upstream release of neutron (not neutron-lib) then i would ahve suggested addign ascond extention for the more effeicnt query but since it has not shipped yet i think we shoudl just fix it and hold the neutorn-lib release until that is done
19:11:16 hyang[m] sean-k-mooney: the neutron api extension and related server code is already released in both neutron-lib and neutron https://review.opendev.org/c/openstack/neutron-lib/+/812617 https://review.opendev.org/c/openstack/neutron/+/811242. Since the behavior using shared=true for SGs is same as other existing resources in Neutron like networks, I'm not sure if Neutron team will want to change it (and change for all the resources apis).
19:12:34 opendevreview Dmitrii Shcherbakov proposed openstack/nova-specs master: Integration With Off-path Network Backends https://review.opendev.org/c/openstack/nova-specs/+/787458
19:41:49 opendevreview Dmitrii Shcherbakov proposed openstack/nova-specs master: Integration With Off-path Network Backends https://review.opendev.org/c/openstack/nova-specs/+/787458
19:50:43 Zer0Byte alright
19:50:46 Zer0Byte i found the issue
19:51:31 Zer0Byte @sean-k-mooney i have two regions and by mistake someone when provisioning put placement on the second region as regionone
19:51:39 Zer0Byte now the issue is solved
19:51:47 Zer0Byte also changed the endpoints url
19:52:10 Zer0Byte but nova-agent is still trying to connect to second region placement
20:09:41 Zer0Byte found the bug
20:09:44 Zer0Byte =)
21:04:30 opendevreview Artom Lifshitz proposed openstack/nova master: api-ref: server rescue adminPass injection is conf-dependant https://review.opendev.org/c/openstack/nova/+/818022
21:19:46 opendevreview Daniel Speichert proposed openstack/nova master: conf: add [libvirt]/no_compression_image_types = qcow2 https://review.opendev.org/c/openstack/nova/+/817995
21:25:53 opendevreview Daniel Speichert proposed openstack/nova master: conf: add [libvirt]/no_compression_image_types = qcow2 https://review.opendev.org/c/openstack/nova/+/817995
#openstack-nova - 2021-11-16
00:59:36 sean-k-mooney hyang[m]: its merged its not relaease yet
03:52:46 opendevreview melanie witt proposed openstack/nova master: Poison usage of eventlet spawn_n() in tests https://review.opendev.org/c/openstack/nova/+/818042
07:10:12 EugenMayer It seems like the first boot of an instance (with cloud-init) has diffenret results in network then all the other after that. Is that desired?
07:47:40 EugenMayer am I right that using --user-data has 2 flavours, using it with --config-drive true will expect to have meta-data alike content there, without using config-drive yaml based cloud-init files are expected. So the same --user-data for 2 different subsystems on init?
07:48:12 EugenMayer (reading https://docs.openstack.org/nova/queens/user/config-drive.html)
08:03:22 gibi_ o/ morning nova
08:40:04 bauzas good spec review day, Nova
09:13:10 opendevreview Merged openstack/nova-specs master: Repropose flavour and image defined ephemeral storage encryption https://review.opendev.org/c/openstack/nova-specs/+/810867
09:33:05 jengbers Morning, I have been searching OpenStack history, but I haven't been able to find why it is not possible to add existing instances to server groups. Does anyone know if that is just never discussed or if there is a fundamental technical problem?
09:33:05 jengbers We have been adding servers to server groups by changing the database and the scheduler has always handled this well.
09:38:38 gibi jengbers: the problem is consistency. When you add a server to a group then you can create a situation when the group membership and the actaul placement of the instance contradicts
09:38:54 gibi and the question is what to do then
09:39:01 gibi a) move the instance to restore consistency
09:39:10 gibi b) reject the addition of the instance to the group
09:40:11 gibi c) allow temporary inconsitency and let the next move operation on the instance fix the group
09:40:34 gibi I think we never agreed which direction to take
09:43:05 sean-k-mooney there was a propsoal to extend it recently to allow this but the suggestion then was to have teh add do migration which i dont thinkis the right approch
09:43:37 sean-k-mooney its particaly a probelm for the affinity policy since that is more likely to break then the anti affinity policy
09:48:17 kashyap bauzas: I might not be able to make the meeting today; I see it's at 17u CET :-(
09:48:27 kashyap Morning, BTW
09:48:56 bauzas kashyap: ah ok, no worries then
09:49:25 bauzas kashyap: just add notes to your specless ask in the agenda, so we can discuss there
09:50:03 kashyap bauzas: Yep; doing that now
09:50:56 kashyap bauzas: If there are any questions, you can ask me here, I'll answer when I'm back later in the evening. I'm away from 17-18u CET
09:52:48 jengbers I guess option b) would be the least surprising.
09:59:31 gibi jengbers: with option b) the problem is that then the user first somehow move the instance to the proper place (but the user has no tool for it) then add it to the group
10:00:34 opendevreview Merged openstack/nova-specs master: Store and allow libvirt instance device buses and models to be updated https://review.opendev.org/c/openstack/nova-specs/+/810235
10:01:17 sean-k-mooney gibi: if you have the correct wiegher enabel i belvie you can ask for an isntnace to be created on the same hos tor a different host to a specific instance via a scheduler hint
10:01:44 sean-k-mooney its certenly requires a lot of knoladge of nova and the instance to do correctly
10:02:18 gibi sean-k-mooney: if you use SameHost / DifferentHost filters then you don't need server groups
10:02:21 sean-k-mooney you cant a s a normal use cold migrate to the same host or simialrly aline them all
10:02:35 sean-k-mooney gibi: ya that is also kind of true
10:02:53 gibi they implement similar logic but a very different way :)
10:03:03 sean-k-mooney yep
10:03:21 sean-k-mooney i do wish we had ways to make server groups more useful
10:03:39 sean-k-mooney but its kind fo hard to extend them for the reason above
10:03:59 gibi sean-k-mooney: for that we need to solve jengbers' problem and also extend the logic to support multiple groups per instance (or nested groups
10:04:02 gibi )
10:04:16 gibi both is painfully missing but hard to solve
10:05:23 gibi bauzas: I'm done with the spec sweep. I could not really comment on the ironic one https://review.opendev.org/c/openstack/nova-specs/+/815789 and it seems nobody commented yet
10:05:37 gibi the rest of the specs has feedback
10:05:40 sean-k-mooney ya. on second tought we should have "server aggreates" in parallel to server groups. you knwo so we can aggreate servers and give that aggreate of servers a name and even have some metadta that can be shared like this server it the primary of the aggreate and just not have it related to vm placement at all
10:05:46 bauzas gibi: I still have 3 specs to look at
10:05:55 sean-k-mooney that way we can pretend server groups dont exist :)
10:05:55 bauzas gibi: but OK, and thanks for the fish
10:06:24 gibi sean-k-mooney: :D
10:14:06 jengbers gibi, sean-k-mooney: If it was only possible for admins, that could work, because they can also migrate servers, but for users it seems quite hard.
10:15:42 gibi jengbers: yeah that could work. Feel free to propose a spec about the new API to get wider discussion around it
10:16:26 jengbers On the other hand, they can power of and start an instance. I guess that would mean it is started on a different hypervisor.
10:20:54 sean-k-mooney really there are 2 paths we could take. 1 allow normall user ask nova to cold/live migrate an instance to be consitent with a server group that it is currently not a member of and then allow them to add the server to the group after, rejecting the request if the policy is volated
10:21:33 sean-k-mooney or 2 we can have the server group add triger the migration as part fo the request
10:24:39 kashyap Is this failing for anyone else too?
10:24:41 kashyap tempest.api.compute.admin.test_live_migration.LiveAutoBlockMigrationV225Test.test_live_migration_with_trunk [108.238299s] ... FAILED
10:25:41 gibi kashyap: could you link the test run?
10:25:56 kashyap gibi: https://zuul.opendev.org/t/openstack/build/632f8ed30e9a4a04a32648843f227ef3
10:25:58 jkulik we've downstream extended the server-groups api to allow adding servers after the fact. we opted for not allowing to add servers if this would be against the server-group's rules
10:25:59 gibi looking
10:27:53 gibi kashyap: I think you got hit by https://bugs.launchpad.net/neutron/+bug/1940425
10:28:06 jkulik it helps customers if they already spawned an instance and forgot the server-group and now want to spawn another instance in some affinity to the existing one
10:28:08 gibi the stack trace is the same
10:28:42 gibi jkulik, jengbers: so both of you would like the same behavior, you should team up proposing this upstream :)
10:28:46 kashyap gibi: Oh, thank you
10:29:10 kashyap gibi: Now what? ... Should I do a "recheck 1940425"?
10:29:25 kashyap Or pray to the ju-ju at the bottom of the sea? Or...
10:29:26 gibi kashyap: yepp, recheck bug 1940425
10:29:46 jkulik https://github.com/sapcc/nova/commit/7220be3968ee1dd257c9add88228cc5bb9857795 is the main commit downstream for us
10:30:16 jkulik gibi: yes, we talked internally already about proposing this upstream, but small team, much work :/
10:30:18 gibi kashyap: I added your run to the bug maybe that way we can get attention to the failure as it is still happening
10:30:21 kashyap gibi: Thx for the quick spot
10:31:03 gibi jkulik: no pressure, I know that type of frustration
10:35:46 bauzas jkulik: we tried discussing this in the past upstream, but operators are very afraid by the races conditions it creates
10:36:39 bauzas jkulik: problem is, in a distributed service model like Nova, you can't get a valid answer whether you can do it, as when you validate, you don't ask the nova-compute service
10:37:48 jkulik bauzas: that reminds me ... we wanted to change the DB to disallow having a server in multiple server-groups to help with races. we haven't done that, yet. thanks :D
10:38:20 bauzas in theory we should hold new instance creations per compute once you ask for adding a new instance to the group
10:40:02 jkulik our problem is a little different, still, as we use VMware and not libvirt. thus, we have a lot of hidden hypervisors as nova only sees the cluster. therefore, hard anti-affinity doesn't really matter for us that much
10:40:48 jkulik customers want to make sure they run on different hypervisors and thus we sync the server-groups to the VMware clusters. VMware then migrates VMs around to make sure the rules apply.
10:42:46 jkulik i.e. most of our customers depend on soft-anti-affinity, which is a Weigher in nova-scheduler anyways
11:43:54 kashyap bauzas: Alright, added it to the Open Discussion here: https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
11:44:48 opendevreview Rajat Dhasmana proposed openstack/nova-specs master: Add spec for volume backed server rebuild https://review.opendev.org/c/openstack/nova-specs/+/809621
12:10:11 dmitriis gibi: tyvm for the feedback
12:10:50 gibi dmitriis: you are welcome. it is a well written spec, thanks for putting in the effort
12:22:45 sean-k-mooney i dont know if i set send on my respocne to the last verions of it

Earlier   Later