| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-07-25 | |||
| 09:58:44 | stephenfin | shoot | |
| 09:59:13 | bhagyashris | stephenfin: 1. 1. I am able create the pinned instance using old way on your patches. you can see the details which steps I have followed to create the instance here http://paste.openstack.org/show/754840/ | |
| 09:59:40 | bhagyashris | stephenfin: 2. I have also checked few scenarios and there I saw some issues. you can see here http://paste.openstack.org/show/754841/ | |
| 10:01:09 | stephenfin | Yeah, I'd expect it to consume both VCPU and PCPU because I don't have the handling code you do. Your implementation is better in that regard | |
| 10:01:33 | stephenfin | The other two scenarios are interesting. I wonder what I'm hitting there | |
| 10:01:41 | bhagyashris | stephenfin: yeah it's handling this case .... | |
| 10:02:24 | bhagyashris | stephenfin: I know where is point that's reporting wrong inventory | |
| 10:04:11 | stephenfin | Oh yeah? | |
| 10:04:29 | bhagyashris | Here https://review.opendev.org/#/c/671793/4/nova/virt/libvirt/driver.py@6852 if I set [compute] cpu_dediacted_set then it's report VCPU resources as well | |
| 10:05:58 | bhagyashris | stephenfin: because in self._get_vcpu_total() method you are checking if vcpu_pin_set , elif CONF.compute.cpu_shared_set else all the host_cpus | |
| 10:06:26 | stephenfin | Ah, yes. So before thatfallthrough case I need a final conditional to check if cpu_dedicated_set is set and return nothing if so | |
| 10:06:46 | stephenfin | And ditto for the '_get_vcpu_total' method | |
| 10:06:55 | bhagyashris | yes | |
| 10:07:32 | bhagyashris | stephenfin: I will keep testing and review your patches | |
| 10:07:57 | stephenfin | bhagyashris: The one I'm most interested in your thoughts on is https://review.opendev.org/#/c/671800/7/nova/objects/numa.py | |
| 10:08:22 | stephenfin | Because I think that and the changes to InstanceNUMACell are the biggest differences we have | |
| 10:09:45 | stephenfin | I don't know what we do with old NUMACell objects. For those, 'cpu_usage' can contain usage of either pinned (PCPU) or unpinned (VCPU) instance vCPUs | |
| 10:09:46 | bhagyashris | stephenfin: yeah same question was in my mid | |
| 10:09:51 | bhagyashris | mind* | |
| 10:09:54 | stephenfin | and we don't ever rebuild the objects from scratch | |
| 10:10:20 | stephenfin | instead, we use that numa_usage_from_instances function to add or subtract usage based on a provided instance NUMA topology | |
| 10:11:10 | stephenfin | I'm thinking it might make sense to start retrieving all instances associated with a host and building the host NUMA topology object from scratch each time | |
| 10:11:28 | stephenfin | but that would involve a join on the instance extra table in some places | |
| 10:14:26 | bhagyashris | stephenfin: yeah that is one option | |
| 10:15:40 | bhagyashris | stephenfin: what's your opinion about my change I mean I made change in both the InctanceNUMAToplogy and host NUMAToplogy | |
| 10:16:49 | stephenfin | Yeah, as noted I'm not sure if it's necessary yet. We have the 'cpu_policy' field on that object so we're already able to tell if 'cpuset' describes VCPUs or PCPUs | |
| 10:17:16 | stephenfin | That will change when we support both types in the same instance, but we're doing that separately | |
| 10:17:26 | stephenfin | Speaking of which, I need to review that spec again today | |
| 10:17:57 | stephenfin | So I don't mind having it, but I think it might be unnecessary for now and possibly make things a little more complicated than necessary | |
| 10:19:51 | bhagyashris | stephenfin: okay, But what I thought is anyways we are going to support both the VCPU and PCPU in future so that will not cause any problem even if we keep now and I have added the api and scheduler check that dont allow both the PCPU and VCPU in one request | |
| 10:20:18 | bhagyashris | stephenfin: so in future there will be just matter of removing that check | |
| 10:20:31 | stephenfin | Yup, I get that. Maybe it makes sense. I haven't really parsed how much complexity it adds so maybe it's not an issue | |
| 10:20:45 | stephenfin | I just wanted to highlight that it wasn't 100% necessary yet, if that makes sense | |
| 10:20:49 | sean-k-mooney | well we will need to add a filed to store teh mask of pinned cores | |
| 10:20:55 | sean-k-mooney | which we should not add until we need it | |
| 10:21:26 | sean-k-mooney | e.g. we should not make object changes that would only be required when we allow mixed instances | |
| 10:21:34 | sean-k-mooney | until we support that | |
| 10:21:42 | stephenfin | Yeah, that's my gut feeling too | |
| 10:21:55 | stephenfin | YAGNI or something like that | |
| 10:22:33 | bhagyashris | stephenfin, sean-k-mooney: ok. | |
| 10:23:57 | stephenfin | bhagyashris: On the plus side, it should be very easy reuse it if/when that spec to allow mixed instances gets merged, so that's a win :) | |
| 10:24:50 | bhagyashris | stephenfin, sean-k-mooney: means for now we will consider the cpuset only | |
| 10:24:58 | bhagyashris | stephenfin: ok | |
| 10:26:00 | bhagyashris | stephenfin: and one more point we are going to allow the new syntax of flavor extra specs like "resources:PCPU=<no of cpus>" right? | |
| 10:26:45 | stephenfin | I think we decided we would have to, yes | |
| 10:26:53 | stephenfin | Though it wouldn't be the preferred option, of course | |
| 10:27:35 | bhagyashris | stephenfin: and I saw that this is not yet implemented in your series of patches ... and I have implemented that so may be we can use that code | |
| 10:28:04 | stephenfin | bhagyashris: I'm working on rebasing your stuff into my series (keeping authorship, of course) to do just that now :) | |
| 10:30:00 | bhagyashris | stephenfin: also the upgrade and reshape part is not in your series of patches and I have submitted the patche https://review.opendev.org/#/c/672224/1 to do so . I will address all your review comments on it and will upload the patch ( this is priority work for me) | |
| 10:32:00 | bhagyashris | stephenfin: now only the part remaining is which one will be the better option this one https://review.opendev.org/#/c/672223/1 or this one https://review.opendev.org/#/c/671801/7 | |
| 10:34:08 | stephenfin | Yeah, pretty much | |
| 10:34:19 | bhagyashris | stephenfin: I guess this one will be the better option ttps://review.opendev.org/#/c/672223/1 because it's simple and also as mentioned in spec that we will used scheduler profiler https://review.opendev.org/#/c/555081/28/specs/train/approved/cpu-resources.rst@451 | |
| 10:34:23 | stephenfin | I'm still on the fence, personally | |
| 10:34:42 | stephenfin | Yeah, definitely a lot less work there | |
| 10:35:11 | stephenfin | The only thing is that rewriting extra specs on the user feels a little wrong. I'm not sure why | |
| 10:35:18 | stephenfin | Assuming these are persisted somewhere | |
| 10:36:32 | bhagyashris | stephenfin: yeah, but it looks simple that and less work and there is no overhead of conf option as well | |
| 10:37:08 | stephenfin | I think we still need the config option, no? | |
| 10:37:46 | stephenfin | Otherwise how do we say "don't start converting these extra specs yet because I don't have enough hosts reporting PCPU inventory" ? | |
| 10:39:51 | bhagyashris | stephenfin: ohh ok | |
| 10:40:19 | stephenfin | I'm not crazy, right? We do need an option for that, yeah? | |
| 10:40:20 | bhagyashris | stephenfin: will wait for others opinion then | |
| 10:45:08 | bhagyashris | stephenfin: TO DO is 1. the aliasing of flavor extra spec 2. upgrade related stuff is remaining ... out of that for first to do will wait for other opinion and I will take the upgrade part on high priority | |
| 10:45:39 | bhagyashris | stephenfin: what's your opinion? | |
| 10:49:19 | openstackgerrit | Maksim Malchuk proposed openstack/nova stable/queens: fix cellv2 delete_host https://review.opendev.org/672690 | |
| 11:01:01 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Follow-up for I2936ce8cb293dc80e1a426094fdae6e675461470 https://review.opendev.org/672669 | |
| 11:01:02 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Rename exception argument https://review.opendev.org/671795 | |
| 11:01:02 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Start reporting PCPU inventory to placement https://review.opendev.org/671793 | |
| 11:01:03 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hardware.get_host_numa_usage_from_instance' https://review.opendev.org/671797 | |
| 11:01:03 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Remove unused function parameter https://review.opendev.org/671796 | |
| 11:01:04 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hardware.instance_topology_from_instance' https://review.opendev.org/671799 | |
| 11:01:04 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'hardware.host_topology_and_format_from_host' https://review.opendev.org/671798 | |
| 11:01:05 | openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Split NUMA object tests https://review.opendev.org/672336 | |
| 11:01:05 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Rework 'hardware.numa_usage_from_instances' https://review.opendev.org/672565 | |
| 11:01:06 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support translating CPU policy extra specs, image meta https://review.opendev.org/671801 | |
| 11:01:06 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800 | |
| 11:01:07 | openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available' https://review.opendev.org/672693 | |
| 11:08:39 | openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Integrate 'sphinx.ext.imgconverter' https://review.opendev.org/665693 | |
| 11:08:39 | openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP! docs: Rework nova console diagram https://review.opendev.org/660147 | |
| 11:08:39 | openstackgerrit | Stephen Finucane proposed openstack/nova master: objects: Remove ConsoleAuthToken.to_dict https://review.opendev.org/652970 | |
| 11:20:13 | slaweq | sean-k-mooney: hi again | |
| 11:20:22 | sean-k-mooney | o/ | |
| 11:20:50 | openstack | Launchpad bug 1836642 in neutron "Metadata responses are very slow sometimes" [High,Confirmed] - Assigned to Slawek Kaplonski (slaweq) | |
| 11:20:50 | slaweq | sean-k-mooney: again about https://bugs.launchpad.net/neutron/+bug/1836642 - I replied to Your last comment | |
| 11:20:54 | slaweq | can You check it? | |
| 11:21:11 | sean-k-mooney | ya i saw | |
| 11:22:52 | sean-k-mooney | what i think happened there was the first call took 13 seconds to complete a second call was recived while the first call was completeing and that took 2 secons since the first call hand not completed | |
| 11:23:10 | slaweq | sean-k-mooney: no, first call took 2 seconds | |
| 11:23:14 | slaweq | and second took 13 | |
| 11:23:30 | slaweq | first call is always for /instance-id | |
| 11:23:44 | sean-k-mooney | ok its proably a cache miss then | |
| 11:23:52 | slaweq | and that would be retried if it would took more than 10 seconds - it's how this ec2-metadata script from cirros works | |
| 11:24:17 | slaweq | and second call (after first was completed fine) was for public-keys and this took 13 seconds | |
| 11:24:36 | slaweq | and this isn't retried by ec2-metadata script so it failed | |
| 11:25:10 | sean-k-mooney | right ok | |
| 11:26:01 | slaweq | sean-k-mooney: so I think that it's like that because each of those requests are processed by different worker thus for each worker data isn't cached | |
| 11:26:24 | slaweq | sean-k-mooney: could it be like that? | |
| 11:26:57 | sean-k-mooney | i think we are using memcahce to chage it but if its in porcess yes | |
| 11:27:08 | sean-k-mooney | if its memcache it still takes a while to propagate | |
| 11:27:36 | sean-k-mooney | those request are ~500 ms apart | |