Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-05-11
15:41:07 dansmith in line with our "we don't schedule at capacity" project goal
15:42:35 gibi dansmith: so if the rpc call reports that the image is not cached, then we would add 2x disk space the allocation candidate query, but only allocate 1x disk space on the selected host for the instance, then on the compute side the image cache code would allocate the other 1x disk space in placement for the cache
15:42:42 dansmith no
15:42:48 dansmith we've already done the a-c query at that point
15:43:02 dansmith we'd just use that to advise us which of the a-c are valid
15:43:11 gibi ahh yeah, you have to now which host you send the rpc call
15:43:14 dansmith right
15:46:14 gibi the weigher thing is good for big deployments but will fall short for edge. As far as I understand my downstream report is from a really small edge site close to capacity. :/
15:47:40 dansmith indeed, although I think I'd argue that for highly constrained situations the separate filesystem is the right approach there anyway, given the (a) usually constrain-able image sets for edge and (b) the need to avoid the race that we'll have in some form anyway
15:47:46 dansmith but yep, it's not a great answer for someone that just wants it to work ideally
15:48:29 gibi I will definitely suggest the separate partition for now as I feel whathever solution we come up with (if any) it will not be backportable
15:48:38 dansmith for sure
15:49:53 dansmith fwiw, making a pre-call to cache_images(background=True) would theoretically give us some lower time-to-boot performance in other cases
15:50:10 dansmith I'm really not sure whether that's a terrible idea or not, but it's an interesting thought
15:50:30 dansmith similar to the cyborg case of starting the programming at bind time from the conductor in parallel to the rest of the instance bringup
15:50:36 gibi it would have a side effect to cache image to a compute that otherwise will not be selected
15:51:18 gibi hm or not
15:51:31 dansmith in the tight case you mean right? We'd start caching an image on a host that the scheduler was going to exclude anyway, which is true
15:52:11 gibi yeah for the thight case when the image would fit into the cache but the instance root disk would not any more
15:52:16 dansmith yep, for sure
15:52:24 gibi but that really tight
15:53:54 gibi what if for thight cases we allow disabling the cache entirely? it is thight so no space for cache
15:54:30 dansmith doing that would require a substantial resdesign of the whole image backend for libvirt I think
15:55:03 gibi is it because we assume that there is a backing file for the root fs image which happen to be the cached image?
15:56:41 dansmith I'm actually not sure what happens if you configure qcow2 and "flatten_images" actually, I'd have to look
15:56:50 dansmith that might have the same effect, I'm not sure
15:59:21 gibi do you mean force_raw_images conf option?
16:00:26 gibi or use_cow_images = False? or a linear combination of the two :)
16:00:44 markguz_ Hi nova folks. i have an instance that failed that i migrated (not live) to another compute host, but upon restarting it errors with
16:00:45 markguz_ Unsupported VIF type binding_failed convert '_nova_to_osvif_vif_binding_failed'
16:00:58 dansmith gibi: yeah there's some combination that results in full flattening, but I'm not sure what they are
16:01:10 dansmith gibi: not sure if tat actually results in the cache image going away, or getting copied or what
16:01:24 dansmith actually, as to be the latter I think since we have to expand the size of it
16:01:26 markguz_ i've tried the various solutions found via google but non work
16:01:46 dansmith so you probably end up with 2x the space initially at least, and then you'd need to immediately purge the original or something
16:02:22 gibi dansmith: ack
16:05:47 gibi dansmith: hm even when the flat backend copies the image to raw it does update the cache to keep the base image https://github.com/openstack/nova/blob/d6450879c7f7dd19366b6f002301fbbf87918026/nova/virt/libvirt/imagebackend.py#L585
16:08:05 gibi dansmith: anyhow thanks for your thoughts I have to drop today soon so I will add some summary of this discussion to the bug.
16:10:06 dansmith gibi: yeah, that's what I was thinking above when I said "but then you'd need to immediately purge"
16:10:16 dansmith gibi: the cache is trying to be a cache
16:12:45 gibi yeah, I see now
16:23:26 openstackgerrit Takashi Natsume proposed openstack/nova master: Remove six.reraise https://review.opendev.org/726898
16:30:14 gibi lyarwood, dansmith: updated the bug 1878024 with what we talked about above.
16:30:14 openstack bug 1878024 in OpenStack Compute (nova) "disk usage of the nova image cache is not counted as used disk space" [Undecided,New] https://launchpad.net/bugs/1878024
16:30:24 dansmith cool
16:30:57 gibi and now I go and bake some bread for dinner
16:31:04 gibi see you tomorrow
18:15:35 openstackgerrit Merged openstack/nova master: Support for --force flag for nova-manage placement heal_allocations command https://review.opendev.org/715395
18:15:48 openstackgerrit Merged openstack/nova stable/queens: Include only required fields in ironic node cache https://review.opendev.org/724862
18:16:01 openstackgerrit Merged openstack/nova stable/queens: Lowercase ironic driver hash ring and ignore case in cache https://review.opendev.org/723054
18:16:10 openstackgerrit Merged openstack/nova stable/rocky: Add config option for neutron client retries https://review.opendev.org/722819
18:16:27 openstackgerrit Merged openstack/nova master: Suppress remaining policy warnings in unit tests https://review.opendev.org/726272
18:22:04 markguz_ anyone know how to get out of Unsupported VIF type binding_failed convert '_nova_to_osvif_vif_binding_failed' hell?
19:35:26 openstackgerrit Harshavardhan Metla proposed openstack/nova master: [Nova] Add reference to Placement installation guide https://review.opendev.org/726936
19:42:27 markguz_ for anyone that's interested setting neutron.ml2_port_bindings.vif_type to "ovs" in the database fixed this for me
19:43:35 sean-k-mooney markguz_: if you get that error its because neutron failed to bind the port
19:43:47 markguz_ i know
19:43:50 sean-k-mooney which normally means there was an error on cthe compute node
19:44:10 markguz_ but once it happens it seems next to impossible to fix it via the normal methods.
19:44:46 sean-k-mooney markguz_: you fix it by setting the host filed to "" or "none" then back to the original hostname
19:45:09 markguz_ the instance will not boot due the "binding_failed" being written into the vif_type field
19:45:33 markguz_ sean-k-mooney: or by updating that field i just mentioned in the neutron db
19:46:02 sean-k-mooney yes but you can do "openstack --os-cloud=admin port set --host none baf2b165-797b-4305-bc6b-5b63250b890d" follow by " openstack --os-cloud=admin port set --host workstation baf2b165-797b-4305-bc6b-5b63250b890d"
19:46:09 sean-k-mooney to do it from the api without db hacking
19:46:36 sean-k-mooney it will actully cause port binding to happen properly recalualting the correct values
19:46:41 markguz_ sean-k-mooney: ok. thanks for that
19:47:01 sean-k-mooney so i did that yesterday becauses i was swaping form the iptables firewall dirver to the ovs one
19:48:18 markguz_ this happened to me when i did a non-live migrate of a shutdown instance to a new host
19:48:42 sean-k-mooney hum it should not happen in that case
19:49:10 sean-k-mooney something obviosly went wrong there should be an error in the neutron server log
19:49:46 markguz_ yeah. haven't had time to deep dive. Was focussed on getting the instance back online
19:50:11 sean-k-mooney ist the instance still in resize_verify or did this happen after that point
19:50:33 sean-k-mooney if you had not confirmed the migrate/resize then you could have reverted
19:51:02 sean-k-mooney if you had then ya db edit or unest and reset the host to rebind the port then hard reboot
#openstack-nova - 2020-05-12
00:43:17 openstackgerrit xuyuanhao proposed openstack/nova master: Optimization the soft-delete logical https://review.opendev.org/724260
01:25:32 openstackgerrit xuyuanhao proposed openstack/nova master: Optimization the soft-delete logical https://review.opendev.org/724260
02:03:13 openstackgerrit Wenping Song proposed openstack/nova master: error may occur when filter scheduler with accelerator https://review.opendev.org/722651
03:16:56 openstackgerrit xuyuanhao proposed openstack/nova master: Optimization the soft-delete logical https://review.opendev.org/724260
04:29:41 openstackgerrit Wenping Song proposed openstack/nova master: error may occur when filter scheduler with accelerator https://review.opendev.org/722651
05:44:08 openstackgerrit Harshavardhan Metla proposed openstack/nova master: [Nova] Add reference to Placement installation guide https://review.opendev.org/726936
06:38:17 gibi good morning Nova
06:47:33 ikla how do I passthrough multiple pci devices in nova?
06:53:36 gibi ikla: https://docs.openstack.org/nova/pike/admin/pci-passthrough.html#configure-a-flavor-controller
07:00:54 sean-k-mooney ikla: you just create a flavor with multiple alia's referenced
07:09:56 ikla I have rocky
07:10:37 jkulik hi, is there something like sub-flavors somewhere on nova's roadmap? Having a flavor that maps to different specific flavors per hardware generation would be great.
07:16:01 ikla do I need to define the devices in the nova controller?
07:18:55 gibi ikla: the alias needs to be defined both on the controller and on the computes, the passthrough_whitelist needs to be defined on each compute
07:19:59 gibi jkulik: I don't know about such feature on the roadmap. I think you can implement that top of nova
07:20:27 ikla do I have multiple lines with passthrough_whitelist ? or can I do it all on one line?
07:21:51 bauzas good morning nov
07:21:57 bauzas err, nova even
07:22:13 gibi ikla: you can have a full json dict, see the config doc https://docs.openstack.org/nova/rocky/configuration/config.html#pci.passthrough_whitelist
07:22:26 gibi ikla: or even a list of such dicts
07:22:36 gibi bauzas: o/
07:25:23 ikla same for alias?
07:26:12 gibi ikla: I think alias is a single dict, so you have to have multiple line of alias = {...} in your config to have multiple aliases
07:28:24 ikla thanks
07:28:33 ikla passthrough cannot have multiple lines?
07:29:09 jkulik gibi, by on top of nova, do you mean on the client? Even if we would control the client, which we don't necessarily, we would have to decide on the precise flavor before going into scheduling afaics. So if a user doesn't care about the hardware version, we would have to make a decision anyways instead of taking "what's free" automatically.

Earlier   Later