Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-01-28
00:15:21 dansmith there's compute at boot time, so you can create a newer flavor, you just need to refuse to boot an instance from it when there are no computes able to handle the request
00:16:40 efried right
00:17:16 efried we just need to be careful not to enforce a new rule that strictifies something until we're sure all the computes are upgraded. Kind of thing.
00:17:51 dansmith not sure what you mean exactly..
00:18:28 dansmith you want to ban creation of the flavor with something if some computes are still old? that's definitely okay if it's appropriate, just not required if it's a spec that just specifies something about the instance
00:20:11 efried I'm saying if we decide to, I don't know, deprecate and remove the 'shared' hw:cpu_policy, it would be reasonable to think you could change the validator accordingly to bounce new flavors with hw:cpu_policy=shared. But you can't, because old computes can (and should be allowed to) still handle 'shared'.
00:20:42 efried (I know that's not a good example, just trying to explain the issue.)
00:22:18 sean-k-mooney am we proably would want a config option of some kind to to contol how deprecations are treated
00:22:48 sean-k-mooney so warn by default and make it an error if you opt in.
00:23:14 sean-k-mooney or something but i thnk we can cross that bridge when we have a example that requires it
00:24:11 efried Well, I definitely think we should have a granny switch that turns the whole validator on or off.
00:24:23 efried I don't know about per-key, if that's what you're suggesting.
00:24:42 sean-k-mooney i was more thinkg a strict vs compatiblity mode thing
00:24:55 sean-k-mooney rather then per key
00:26:34 sean-k-mooney if we wanted to deprecate something then we would mark it as such in the validtor and consier it an error in stict mode but just warn on it in compatibliyt mode. then if we drop support entirely we can graduate to always an error becasue we know it will not be supprotin in n+X releases
00:26:52 sean-k-mooney that said im not sure we need to do that
00:27:15 sean-k-mooney if the extra spec is not support by a compute anymore its just ignored
00:27:51 dansmith efried: right so that's an example where controlling the flavor definition based on compute levels would be good.. where changing the flavor impacts the way computes behave, as opposed to just something that affects the instance like a new disk controller or something
00:28:40 efried yikes.
00:29:12 dansmith yikes/
00:29:15 dansmith I'm agreeing with you
00:29:24 dansmith I mean, I think :D
06:29:53 HagunKim Hello, Nova! I have a question. Sometimes my VMs are deleted unexpectedly. What conf or log should I check??
06:29:53 HagunKim Here is nova-compute log when a vm is deleted. I removed date, req. id, instance id.
06:29:53 HagunKim INFO nova.compute.manager [instance: intance id] Terminating instance
06:29:53 HagunKim INFO nova.virt.libvirt.driver [-] [instance: intance id] Instance destroyed successfully.
06:29:54 HagunKim INFO nova.compute.manager [-] [instance: intance id] VM Stopped (Lifecycle Event)
06:29:54 HagunKim INFO nova.compute.manager [instance: intance id] During sync_power_state the instance has a pending task (deleting). Skip.
06:29:55 HagunKim INFO nova.virt.libvirt.driver [instance: intance id] Deleting instance files /var/lib/nova/instances/instance_id_del
06:29:55 HagunKim INFO nova.virt.libvirt.driver [instance: intance id] Deletion of /var/lib/nova/instances/instance_id_del complete
06:29:56 HagunKim INFO nova.compute.manager [instance: intance id] Took 23.99 seconds to destroy the instance on the hypervisor.
08:29:07 gibi HagunKim: with the request_id from your log I would grep the nova logs to see what other events lead to the deletion of the instance
08:29:22 bauzas good morning Nova
08:31:23 gibi bauzas: good morning
08:31:44 bauzas gibi: how are you ? I was a bit on and off last week
08:34:04 gibi bauzas: thanks I'm pretty OK, spent half of my last week downstream with helping planning some internal REST APIs. Do you still need to spend a lot of time downstream?
08:34:17 bauzas gibi: this week, nope
08:34:26 bauzas working on specs and the cold migration issue
08:34:32 bauzas (for GPUs)
08:34:51 bauzas if you have things to be reviewed by me, tell me
08:35:30 openstackgerrit Merged openstack/nova-specs master: support live migration with virtual persistent memory https://review.opendev.org/695863
08:36:17 gibi bauzas: I think there is a small test issue in your placement audit https://review.opendev.org/#/c/670112 but besides that it is pretty close. And the placement fixture enahncement top of that needs a squash / respin
08:37:20 gibi bauzas: on my side the qos live migration needs some review. The bootom patch was commented by dansmith so there we need a re-review from dansmith but you can be the second review if you wish https://review.opendev.org/#/c/702261/
08:37:23 bauzas gibi: sure, lemme know which test issue you found :)
08:37:42 bauzas and /me clikcs on the qos live migration series :)
08:38:55 gibi bauzas: there is two test failure in the functional test in the placement audit patch https://834f0f74916383443a4f-1960c95d24ebbbc2bc76dbab48507616.ssl.cf1.rackcdn.com/670112/13/check/nova-tox-functional-py36/61deb16/testr_results.html
08:39:01 gibi bauzas: thanks
08:39:16 gibi bauzas: you can also hit me with patches to review
08:39:49 bauzas gibi: ah shit, I thought it was a functional test race
08:39:55 bauzas since we had issues last week
08:40:23 bauzas gibi: and sure, but my point is that I'd also like to review specs and changes
08:40:42 bauzas gibi: so if you see other changes that you'd get second eyes...
08:41:23 gibi bauzas: sure I will let you know
09:34:40 openstackgerrit Kobi Samoray proposed openstack/nova master: Avoid fetching metadata when no subnets found https://review.opendev.org/679247
09:44:53 openstackgerrit Kobi Samoray proposed openstack/nova master: Avoid fetching metadata when no subnets found https://review.opendev.org/679247
09:48:04 openstackgerrit Kobi Samoray proposed openstack/nova master: Support large network queries towards neutron https://review.opendev.org/704520
11:12:24 openstackgerrit Stephen Finucane proposed openstack/os-resource-classes master: setup.cfg: Configure 'python-requires' https://review.opendev.org/704533
11:12:28 openstackgerrit Stephen Finucane proposed openstack/os-traits master: setup.cfg: Configure 'python-requires' https://review.opendev.org/704534
11:12:29 openstackgerrit Stephen Finucane proposed openstack/os-traits master: setup.cfg: Remove unused 'build_sphinx' configuration https://review.opendev.org/704535
11:17:00 openstackgerrit Vladyslav Drok proposed openstack/nova master: Make RBD imagebackend flatten method idempotent https://review.opendev.org/704330
11:25:41 openstackgerrit Kobi Samoray proposed openstack/nova master: Avoid fetching metadata when no subnets found https://review.opendev.org/679247
11:27:34 openstackgerrit Kobi Samoray proposed openstack/nova master: Avoid fetching metadata when no subnets found https://review.opendev.org/679247
11:27:52 openstackgerrit Kobi Samoray proposed openstack/nova master: Support large network queries towards neutron https://review.opendev.org/704520
12:01:10 HagunKim gibi: Thanks for your reply. But I found nothing with request_id. I will change log level to debug and see it again.
12:05:18 gibi not even in other nova services like nova api?
12:15:42 bauzas HagunKim: just use the os-instance-actions API
12:16:31 bauzas HagunKim: https://docs.openstack.org/api-ref/compute/#servers-actions-servers-os-instance-actions
12:19:07 HagunKim bauzas: Can I use that api for removed instances?
12:19:22 bauzas "Action information of deleted instances can be returned for requests starting with microversion 2.21."
12:19:26 bauzas :)
12:20:07 bauzas then you'll get the req-ids
12:20:24 bauzas so you can either ask again the same API for a specific req-id or just look at logs
12:20:47 HagunKim Oh I will check. Thank you!
12:33:40 stephenfin yo, sean-k-mooney, you probably want to weigh in on https://review.opendev.org/668656 since I _think_ you're of the same mindset as me RE: not forcing people to use placement constructs in nova land
12:34:15 sean-k-mooney i have in the irc discussions and yes
12:35:12 sean-k-mooney i would be oke with the placement syntax but only if we remove support for placement groups syntax form the flavor and stop doing a direct passthough to the api query effectivly
12:36:00 sean-k-mooney e.g. resouce:* would be ok if and only if we have a layer that parses that and then generate a different query based on the other extraspecs but i prefer the hw:* syntax
12:36:08 openstackgerrit Kobi Samoray proposed openstack/nova master: Avoid fetching metadata when no subnets found https://review.opendev.org/679247
12:40:23 stephenfin my arguments slightly different to that. I'm okay with the placement syntax but I don't want it mixed with the legacy stuff. It's too confusing to reason about
12:43:46 alex_xu sean-k-mooney: stephenfin, if we are going to the vision about placement syntax in the future, will it be hard to deprecated those extra specs we added?
12:45:42 alex_xu I'm just thinking, it is vision, agreed by some people, but it is vision, we don't know when it happend, but people afarid we add more stuff which aren't existing in that vision. just thinking of if we have road to transfer to that vision, then we continue add extra specs, then it won't be a problem
12:47:18 sean-k-mooney alex_xu: i guess the only way i can see moving to placement syntax workin is if we remove the group syntax form the flavor
12:47:42 sean-k-mooney so i for won dont ever see use moving to placemnt syntax unless that happens
12:48:26 sean-k-mooney if the resouces:* requests are coupled to the placemetn toplogy directly that is an upgrade issue
12:49:01 sean-k-mooney that said with an indirection laywer between the placment query and the flavor it would be managable
12:49:11 alex_xu I see
12:49:46 sean-k-mooney but to your question i dont think doing hw:* is a blocker to doing resouces:* in the future
12:50:23 alex_xu but at that time, we need to deprecated some extra sepcs like 'hw:cpu_policy', any problem we deprecated one more more extra spec for 'hw:dedicated_cpu_set'(I forget the name)?
12:50:43 sean-k-mooney we can support that for cpu pinning today but i would continue to recommend using hw:cpu_policy=dedicated over resocues:PCPU=x
12:50:44 alex_xu yea, that is what i'm thinking
12:51:34 alex_xu huaqiang: efried dansmith stephenfin ^
12:51:41 sean-k-mooney ya we would have to deprecate both toghter but 1 vs 2 extra specs is not really a big differece
12:51:45 alex_xu it is time pull everybody togetther again :)
12:52:15 sean-k-mooney we would still have to do an online data migration to update the embeded flavor or force the operator to resize every vm in either case
12:53:02 sean-k-mooney that is when we remove support for hw:cpu_policy if we ever do
12:53:12 alex_xu ah, resize, that is terrible
12:53:48 sean-k-mooney ya its the only way to update the embeded flavor today
12:54:23 sean-k-mooney which is why i dont like the idea of useing the placement syntax direcly as it is copled to the placement toplogy and if that chagne we need to either do a resize or an online data migration
12:55:13 alex_xu yea, i see
12:56:01 sean-k-mooney given we want to move pcpus under numa nodes in the next release or two i worry about relying on the placment syntax currently as that will be a toplogy change

Earlier   Later