| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2020-05-12 | |||
| 07:22:13 | gibi | ikla: you can have a full json dict, see the config doc https://docs.openstack.org/nova/rocky/configuration/config.html#pci.passthrough_whitelist | |
| 07:22:26 | gibi | ikla: or even a list of such dicts | |
| 07:22:36 | gibi | bauzas: o/ | |
| 07:25:23 | ikla | same for alias? | |
| 07:26:12 | gibi | ikla: I think alias is a single dict, so you have to have multiple line of alias = {...} in your config to have multiple aliases | |
| 07:28:24 | ikla | thanks | |
| 07:28:33 | ikla | passthrough cannot have multiple lines? | |
| 07:29:09 | jkulik | gibi, by on top of nova, do you mean on the client? Even if we would control the client, which we don't necessarily, we would have to decide on the precise flavor before going into scheduling afaics. So if a user doesn't care about the hardware version, we would have to make a decision anyways instead of taking "what's free" automatically. | |
| 07:32:29 | ikla | nvrmind :) | |
| 07:34:48 | gibi | ikla: I think passthrough_whitelist also can be used multiple times | |
| 07:36:06 | ikla | does passthrough_whitelist work on rocky or does it need to be prefixed with pci_ ? | |
| 07:37:39 | gibi | ikla: in the [pci] section you can use passthrough_whitelist config | |
| 07:50:49 | openstackgerrit | Harshavardhan Metla proposed openstack/nova master: [Nova] Add reference to Placement installation guide https://review.opendev.org/726936 | |
| 07:59:03 | openstackgerrit | James Page proposed openstack/nova stable/queens: Fix overcommit for NUMA-based instances https://review.opendev.org/726868 | |
| 07:59:03 | openstackgerrit | James Page proposed openstack/nova stable/queens: hardware: fix memory check usage for small/large pages https://review.opendev.org/726867 | |
| 08:19:27 | openstackgerrit | sean mooney proposed openstack/nova-specs master: move implemented specs in ussuri https://review.opendev.org/721278 | |
| 08:19:55 | sean-k-mooney | gibi: sorry ment to do that last week ^ if you want any changes let me know and ill respin the patch | |
| 08:29:20 | huaqiang | stephenfin: There are ~20 patches in bp/use-pcpu-and-vcpu-in-one instance, for roles of py-checks, realtime policy, and fixes for hardware.py | |
| 08:29:37 | huaqiang | these are not depending on each other so much | |
| 08:29:51 | huaqiang | how about let me re-arrange the order | |
| 08:30:40 | huaqiang | and resolving some dependencies with slight change to make it possible to be reviewed parallelly | |
| 08:32:18 | openstackgerrit | Nalini Varshney proposed openstack/nova master: Add migration to make key field type VARBINARY in aggregate_metadata table, https://review.opendev.org/725522 | |
| 08:35:00 | bauzas | gibi: impressive count on open bugs https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New | |
| 08:35:12 | bauzas | gibi: how have you triaged them ? | |
| 08:48:12 | gibi | bauzas: most of the did not have enough information | |
| 08:48:50 | gibi | bauzas: and during the last two weeks we get only about 5 new bug reports so the low inflow helped a lot | |
| 08:49:51 | gibi | sean-k-mooney: thanks, I will check it | |
| 08:50:42 | gibi | jkulik: what you want to achive? Do you want to move users from old HW gen to new HW gen over time? | |
| 08:51:07 | stephenfin | huaqiang: Sure. The only issue with that is that future rebases are harder. If you can keep the existing +2s though, go for it | |
| 08:52:29 | jkulik | gibi, we have different hardware versions in place at the same time, where, because of NUMA and whatnot, flavors differ slightly. We want a user to be able to deploy a "2 TB instance" flavor, even if that means 1.8 on one node and 2.1 on the other. | |
| 08:53:35 | sean-k-mooney | jkulik: that is how it works by default | |
| 08:54:18 | sean-k-mooney | unless you meen 1.8 TB on one and 2.1 TB on the other? in not sure what those number refer too | |
| 08:54:48 | jkulik | sean-k-mooney, the flavor has to map to 1.8 TB on one hardware version and 2.1 TB on the other | |
| 08:55:00 | sean-k-mooney | jkulik: ya that is not allowed | |
| 08:55:01 | jkulik | sean-k-mooney, we just want our users to be able to specify "give me that 2 TB thingy" | |
| 08:55:22 | sean-k-mooney | right but its a 1.8TB thing and a 2.1TB thing | |
| 08:55:31 | sean-k-mooney | its not a 2TB thing in either case | |
| 08:55:43 | jkulik | because it's hard for us to keep hypervisors empty enough to support that size, the user can't really know up-front what hardware is currently free | |
| 08:55:47 | sean-k-mooney | you could use boot form volume and not provide any default root disk | |
| 08:56:09 | jkulik | we're talking RAM here :/ | |
| 08:56:20 | sean-k-mooney | oh i taught you ment disk | |
| 08:56:33 | sean-k-mooney | and the user should know know up front how much ram is free | |
| 08:56:38 | sean-k-mooney | *should not | |
| 08:57:09 | sean-k-mooney | jkulik: it sound like you want to reconfigure the scheduler to pack | |
| 08:57:10 | openstackgerrit | Tony Su proposed openstack/nova-specs master: Re-propose provider-config-file spec for Victoria https://review.opendev.org/725788 | |
| 08:57:16 | jkulik | I don't see how we could tell the user what flavors could currently get deployed via nova's api | |
| 08:57:39 | sean-k-mooney | if you configure the schduler to pack based on ram usage it will try to keep hosts free if it will fit on othet hosts | |
| 08:57:57 | sean-k-mooney | jkulik: well you cant and that is kind fo intentional | |
| 08:57:59 | jkulik | sean-k-mooney, the problem is even bigger for us: we use vmware .. so we have a cluster in the compute node and have to do some magic behind the scenes to convince vmware to spawn this monster | |
| 08:58:20 | sean-k-mooney | it breaks the cloud model if the tenant has a view of the aviaable capacity | |
| 08:58:47 | jkulik | yeah, that's why I wanted a flavor, that can match any of those free hosts, by having sub-flavors ;) | |
| 08:59:00 | sean-k-mooney | jkulik: yep which is not going to happen | |
| 08:59:12 | sean-k-mooney | we kind of had something like that at one point | |
| 08:59:14 | sean-k-mooney | for ironic | |
| 08:59:23 | sean-k-mooney | but that has been gone for a long time | |
| 08:59:57 | jkulik | we'll have to work something out downstream then, thanks for clarifying :) | |
| 09:00:24 | sean-k-mooney | im trying to think if we could use placmenet and custom resouces classes for this | |
| 09:00:48 | sean-k-mooney | but what you are trying to do is going against the grain of how nova is intended to be used ot a degree | |
| 09:00:52 | bauzas | gibi: ack thanks | |
| 09:03:05 | sean-k-mooney | jkulik: im not sure how configurable the vmware dirver is | |
| 09:03:30 | sean-k-mooney | but do you want to reserve space for this giant vms or no? | |
| 09:04:13 | openstack | Launchpad bug 1838309 in OpenStack Compute (nova) "Live migration might fail when run after revert of previous live migration" [Undecided,Won't fix] - Assigned to Vladyslav Drok (vdrok) | |
| 09:04:13 | bauzas | gibi: agreed with Wontfix https://bugs.launchpad.net/nova/+bug/1838309 ? | |
| 09:04:57 | sean-k-mooney | what you could do woudl be to reserve ram on the specifci hosts where you intend to spawn these isntances. then create a custom resouce class for them and a flavor that request 2TB or ram but set resocues:memory_mb=0 resouces:CUSTOM_GIANT_VM=1 | |
| 09:05:00 | jkulik | sean-k-mooney, not in general. we currently run something, that frees up enough space on one note - so we can spawn one. | |
| 09:05:32 | sean-k-mooney | ok so that wont work if you want it to be dynamic then | |
| 09:05:33 | jkulik | yeah, we have something like that. but it has to fit into normal quota for billing | |
| 09:05:47 | tony_su | gibi: This is Tony Su and I am doing provider-config-file re-propose thing. Could you kindly spare some time to review its spec (only minor changes in History and Assignee sections vs. Ussuri version). https://review.opendev.org/#/c/725788/ | |
| 09:06:22 | sean-k-mooney | jkulik: well the flavor.ram value would still be 2TB | |
| 09:06:32 | jkulik | sean-k-mooney, we dynamically create a sharing child-resource-provider for a node, that has the CUSTOM_GIANT_VM resource | |
| 09:07:23 | sean-k-mooney | ok but the issue is you still need to aling the flavor.ram to what that means for that host | |
| 09:07:24 | jkulik | sean-k-mooney, so currently, it works for us. but the flavor's RAM doesn't align with NUMA on all nodes. meaning, we would have to create more flavors. | |
| 09:07:31 | sean-k-mooney | based on the hardware version | |
| 09:07:35 | jkulik | yes | |
| 09:08:12 | sean-k-mooney | jkulik: may i ask why you actully use VM instead of ironic nodes at that scale | |
| 09:08:19 | jkulik | but given that we only free up one node, that could fit 2 TB either because it's 2 TB in size or because it's bigger, the 2 TB are more a rounded value the custerom might request | |
| 09:08:32 | tony_su | gibi: Really appreciate your support and help. | |
| 09:08:58 | jkulik | sean-k-mooney, sure. we were on ironic, but our customers would like to have the automatic failover when a node goes down, that vmware provides. also TCO. | |
| 09:10:25 | sean-k-mooney | i see so they are relying on the plathform for failover rahter then using an orchestration layer like k8s to mage there application | |
| 09:10:39 | ikla | the controller and node doesn't need to have the same device in in for passthrough? | |
| 09:10:40 | sean-k-mooney | so this is very much a pets not cattel usecase | |
| 09:10:50 | jkulik | since the customer can't find out, which precise flavor she can currently deploy via API, it would have been nice to have specific sub-flavors for ~ 2 TB | |
| 09:10:52 | gibi | bauzas, stephenfin: easy +2 https://review.opendev.org/#/c/721278 (ussuri spec move) | |
| 09:10:57 | ikla | my flavor keeps failing when trying to turn on a instance | |
| 09:11:03 | sean-k-mooney | ikla: the alias need to be on the computes and contoler | |
| 09:11:14 | jkulik | sean-k-mooney, yes. definitely pets | |
| 09:11:18 | sean-k-mooney | ikla: the pci whitelist is only needed on the compute | |
| 09:11:27 | ikla | alias yes, device only on compute.. | |
| 09:11:28 | ikla | k | |
| 09:11:52 | sean-k-mooney | ikla: so the contole just uses it to translate the value in the flavor into pci request for schduling | |
| 09:12:20 | openstack | Launchpad bug 1838309 in OpenStack Compute (nova) "Live migration might fail when run after revert of previous live migration" [Undecided,Won't fix] - Assigned to Vladyslav Drok (vdrok) | |
| 09:12:20 | gibi | bauzas: I agree with WontFix https://bugs.launchpad.net/nova/+bug/1838309 | |
| 09:12:29 | bauzas | cool, moving on | |
| 09:13:03 | gibi | tony_su: thanks for taking that feature over. I will look at the spec soon | |
| 09:15:27 | openstack | Launchpad bug 1878024 in OpenStack Compute (nova) "disk usage of the nova image cache is not counted as used disk space" [Undecided,Confirmed] | |
| 09:15:27 | gibi | bauzas: thanks for looking at the new bugs. You have the bug lock, I have yet another downstream originated issue to look at https://bugs.launchpad.net/nova/+bug/1878024 | |
| 09:16:12 | sean-k-mooney | gibi: thats a tricky one | |
| 09:16:40 | sean-k-mooney | in that really we should never fail to boot a vm becasue of a cached image taking up space | |
| 09:17:10 | sean-k-mooney | i.e. i would expect as an operator for the cache to be perurged first | |
| 09:17:25 | sean-k-mooney | so im not sure i would want it to be counted as used | |