| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-08-07 | |||
| 12:11:53 | sean-k-mooney | all the discusstion seams to be on 7 but the patch is on version 11 | |
| 12:14:19 | sean-k-mooney | i think the table for the setting in the image vs flavor is wrong by the way | |
| 12:26:35 | openstackgerrit | Alex Xu proposed openstack/nova master: Add functional tests for resize with VGPU https://review.opendev.org/675106 | |
| 12:27:55 | openstackgerrit | Stephen Finucane proposed openstack/nova master: scheduler: Flatten 'ResourceRequest.from_extra_specs', 'from_image_props' https://review.opendev.org/674894 | |
| 12:27:55 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for translating CPU policy extra specs, image meta https://review.opendev.org/671801 | |
| 12:27:56 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Add reshaper for PCPU https://review.opendev.org/674895 | |
| 12:40:31 | alex_xu | sean-k-mooney: dansmith also has a point about whether we should let the user specific which vcpu is dedicated | |
| 12:43:37 | openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Nova object changes for forbidden aggregates request filter https://review.opendev.org/671072 | |
| 12:45:01 | openstackgerrit | Shilpa Devharakar proposed openstack/nova master: DB API changes to get non-matching aggregates from metadata https://review.opendev.org/671074 | |
| 12:46:18 | openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Add a new request filter to forbidden aggregates https://review.opendev.org/671075 | |
| 12:47:11 | sean-k-mooney | alex_xu: dansmith the reason for being able to define which cpus are dedicated is the same for why we allow it for realtime cpus. we want to be able to say that core x is low performace and can be used for the os/manament interfaces vs core y is high perfomace and can be used for the application | |
| 12:48:03 | openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Enable request filter forbidden_aggregates https://review.opendev.org/667952 | |
| 12:48:28 | sean-k-mooney | alex_xu: dansmith: so yes we do need to allow the user to pass a mask in my opipion and i dont think nova should do any magic to try and calualte it. | |
| 12:48:56 | alex_xu | sean-k-mooney: hint by you, if I want the realtime cpu to be dedicated, then I still need a way to define that. | |
| 12:49:03 | sean-k-mooney | if the user has multiple numa nodes and/or a specific cpu toplogy then they shoudl understand that mapping | |
| 12:49:40 | sean-k-mooney | well that is simple. use the same mask for both | |
| 12:49:50 | alex_xu | yea, otherwise, we will have a magic to mapping realtime vcpu to dedicated vcpu inside nova | |
| 12:49:55 | sean-k-mooney | but really i dont think we should allow realtime core to not be dedicated | |
| 12:50:57 | alex_xu | ah, that make sense. but it still will be a magic | |
| 12:51:02 | sean-k-mooney | if you are declaring a core as realtime it does not make sense for it to be multiplex to a shared host core that other vms can execute on as that will violate any realtiem logic | |
| 12:51:21 | sean-k-mooney | well we just and the bitmasks | |
| 12:51:56 | sean-k-mooney | actuuly xor the bits and check the result is 0 | |
| 12:52:30 | sean-k-mooney | actully no that is also wrong | |
| 12:53:05 | sean-k-mooney | but you get the idea we just need to validate each bit set in the realtime mask is set in the dedicated mask | |
| 12:53:19 | alex_xu | yea | |
| 12:53:23 | alex_xu | i got it | |
| 12:55:18 | sean-k-mooney | to dansmith point however i do belive we discusssed the idea of not exposing a mask in the past and instead doing what he say and just having tally counts of how many of each type you want | |
| 12:55:54 | sean-k-mooney | then evenly distubting them over the numa nodes and ensureing within a numa node the shared cpus are first followed by the pinned ones | |
| 12:56:16 | sean-k-mooney | that fits the paradime where core 0 is often reserved for the os | |
| 12:56:53 | sean-k-mooney | and in a multi numa works the first core on each numa node is often reserved for the os | |
| 12:57:17 | sean-k-mooney | but that does seam like magic that would easily break | |
| 13:00:39 | alex_xu | good to know | |
| 13:01:53 | sean-k-mooney | alex_xu: i saw the irc ping for the patch. what tempeted you to work on that | |
| 13:02:23 | sean-k-mooney | also what time zone are you in. i assume its eitehr quite late or very early for you | |
| 13:05:24 | alex_xu | sean-k-mooney: hah | |
| 13:19:53 | openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Enable request filter isolate_aggregates https://review.opendev.org/667952 | |
| 13:23:58 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: docs: clarify nova migration-list --host option https://review.opendev.org/674950 | |
| 13:23:58 | openstackgerrit | Matt Riedemann proposed openstack/python-novaclient master: WIP: Add --migration-type and --source-compute to migration-list https://review.opendev.org/675117 | |
| 13:28:59 | mriedem | lyarwood: as you're probably aware once these land https://review.opendev.org/#/q/I5e0a43ec59341c9ac62f89105ddf82c4a014df81 we'll be doing stable branch releases | |
| 13:29:35 | sean-k-mooney | mriedem: those are the cve backports right | |
| 13:29:40 | mriedem | yes | |
| 13:29:46 | mriedem | dansmith: can you get this this stein backport? https://review.opendev.org/#/c/672723/ | |
| 13:30:09 | mriedem | melwitt: can you get these stein backports for a series you're familiar with https://review.opendev.org/#/q/topic:bug/1764556+(status:open+OR+status:merged)+branch:stable/stein | |
| 13:30:39 | lyarwood | mriedem: ack yup | |
| 13:30:55 | dansmith | mriedem: jes | |
| 13:31:41 | mriedem | thanks | |
| 13:42:44 | mnaser | can i get a second pair of eyes | |
| 13:42:50 | mnaser | i think there is a nova bug here that we uncovered in OSA | |
| 13:42:52 | mnaser | https://logs.opendev.org/33/660333/10/check/openstack-ansible-deploy-aio_metal-ubuntu-bionic/97d8bc3/logs/host/nova-compute.service.journal-23-20-40.log.txt.gz#_Aug_06_22_51_25 | |
| 13:42:58 | mnaser | nova-compute magically trying to access db? | |
| 13:43:07 | mnaser | (and raising an exception because of that) | |
| 13:45:44 | yonglihe | ||
| 13:47:28 | sean-k-mooney | am i can make time. i saw melwitt had a question when i looked last week about shutdown_unkown. was that resolved | |
| 13:47:37 | mnaser | im trying to do git blame on github and its timing out | |
| 13:47:37 | mnaser | boo | |
| 13:47:44 | mriedem | mnaser: hmm, yeah compute is calling compute.API.stop to stop the instance, which is hitting some code in the compute rpcapi client router to determine the minimum compute version across all cells which it shouldn't need to be doing, b/c you're in a cell | |
| 13:48:18 | mnaser | thats what i figured based on the traceback but i havent done my share of git blame yet to see when this might be started | |
| 13:48:25 | mnaser | https://github.com/openstack/nova/blame/master/nova/compute/manager.py crashes github | |
| 13:48:45 | mriedem | mnaser: https://review.opendev.org/#/c/649197/ | |
| 13:48:56 | sean-k-mooney | mnaser: ya that and the libvirt dirver are two of the files better blamed on the terminal | |
| 13:49:37 | mriedem | i want to say we should be hitting that when compute starts up though, or used to? | |
| 13:49:57 | mriedem | mnaser: do you have the api db configured in nova.conf that you're computes are using? | |
| 13:50:10 | mnaser | mriedem: in this case i believe its an aio so yes it will be there | |
| 13:50:23 | mnaser | in an actual deployment, osa doesnt do this | |
| 13:50:36 | mriedem | even if it's a single node you can run nova-compute with a different config file | |
| 13:50:38 | mriedem | w/o the db creds | |
| 13:50:42 | mriedem | that's what devstack does | |
| 13:50:42 | dansmith | mnaser: is the api db connection set? | |
| 13:50:55 | dansmith | I think that's how it determines if it should use the all-cells or this-cell method | |
| 13:51:14 | mnaser | yeah.. it would be a very 'CI' specific thing if we used 2 different configs, unless we always used 2 configs, but yeah | |
| 13:51:34 | mnaser | if we have to roll out nova-compute.conf and nova.conf -- that's fine, we'll fix that | |
| 13:51:38 | sean-k-mooney | mnaser: well always using two config makes sense in a way | |
| 13:51:40 | dansmith | mnaser: was that aimed at me? | |
| 13:52:01 | sean-k-mooney | it is nice to be able to seperate compute only things for contoler only things | |
| 13:52:05 | mnaser | dansmith: just in general, im wondering if we're doing things wrong (i.e. nova operates with the assumption that api db _should not_ be set in nova-compute) | |
| 13:52:23 | mriedem | i've got to run my kid somewhere, bbiab | |
| 13:52:38 | dansmith | mnaser: the assumption is that computes should not be talking to any db, and thus shouldn't have creds set at all | |
| 13:52:57 | dansmith | mnaser: like, the creds shouldn't even need to be on those systems | |
| 13:53:06 | sean-k-mooney | the compute talkign directly to a db was a legacy feature correect | |
| 13:53:13 | sean-k-mooney | its inteded to go via the conductor | |
| 13:53:18 | mnaser | right, in a 'production' deployment of OSA (where compute != control), the value is not set, but in our aio, the logic is compute is inside the controller group so it gets added | |
| 13:53:41 | mnaser | i agree with you that it should not be set | |
| 13:53:44 | dansmith | mnaser: okay, but it should have a different config which matches what a compute looks like non-aio | |
| 13:54:22 | mnaser | https://github.com/openstack/openstack-ansible-os_nova/blob/master/templates/nova.conf.j2#L196-L212 | |
| 13:54:44 | mnaser | so we only set these if we're running against a host that is not part of nova-compute group, but in this case in aio, every host is so it gets included | |
| 13:54:46 | dansmith | this is obviously failing because of a somewhat fragile assumption about what having api db creds set means, but.. this is why it's good to use realistic configs even in an aio situation | |
| 13:55:27 | sean-k-mooney | mnaser: do ye share config files between services? | |
| 13:55:43 | mnaser | sean-k-mooney: yes, but its templated to be different depending where it ends up getting deployed | |
| 13:55:46 | sean-k-mooney | e.g. i know kolla ansible generate a different nova.conf per nova service even on the same host | |
| 13:56:04 | mnaser | yes we do that too, but they all end up at /etc/nova/nova.conf | |
| 13:56:05 | sean-k-mooney | mnaser: right that is different then kolla ansible | |
| 13:56:17 | sean-k-mooney | right within the contianer | |
| 13:56:25 | sean-k-mooney | but different paths in the host right | |
| 13:56:42 | mnaser | nah its always at /etc/nova/nova.conf here | |
| 13:56:58 | sean-k-mooney | so its alway one file on the host? | |
| 13:57:04 | mnaser | yep | |
| 13:57:15 | mnaser | i mean for me its fine, if from a nova perspective, the assumption that api_db shouldnt be defined, i can fix it inside osa, but if you think that assumption isn't right, i can also fix it inside nova | |
| 13:57:24 | sean-k-mooney | ach well that explains why this does not work then | |
| 13:57:59 | sean-k-mooney | well the assumtion is no db will be defiend in the compute agent's nova.conf | |
| 13:58:23 | sean-k-mooney | all db acess form the comptue agent is expected to go via the cell conductor | |