| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2021-06-23 | |||
| 08:05:01 | lyarwood | morning \o | |
| 08:59:48 | gibi | \o | |
| 09:15:13 | frickler | lyarwood: https://bugs.launchpad.net/nova/+bug/1452641 just came up in #openstack-dev , are you still planning to proceed with https://review.opendev.org/c/openstack/nova/+/579004/ ? (changing ceph mon addresses) | |
| 09:18:14 | lyarwood | frickler: yeah but not as part of that change anymore, I'm writing up a spec at the moment to provide a set of nova-manage commands to allow operators to do refresh this for SHUTOFF instances | |
| 09:18:48 | lyarwood | frickler: the alternative is for users to shelve and unshelve the instances | |
| 09:20:30 | frickler | lyarwood: well from an operator perspective it would be great to have a solution that it transparent to the users and allows them to keep instances running, but I admit that that may not be achievable | |
| 09:21:20 | lyarwood | yeah that's pretty hard if impossible | |
| 09:22:24 | bauzas | will be back around 1130UTC | |
| 09:57:50 | opendevreview | Lee Yarwood proposed openstack/nova master: zuul: Add CentOS 8 stream integrated compute tempest job to gate https://review.opendev.org/c/openstack/nova/+/797616 | |
| 11:58:26 | sean-k-mooney | frickler: the best way to do that might just be to alwasy put the ceph monds behind haproxy or a keepalived vrrp vip | |
| 11:58:42 | sean-k-mooney | frickler: e.g. do not have the actul mon ips present | |
| 11:58:57 | sean-k-mooney | have a sperate one that you can move there instead | |
| 11:59:46 | gibi | sean-k-mooney, bauzas: I've left my view about vgpu in https://review.opendev.org/c/openstack/nova-specs/+/780452 | |
| 12:00:17 | bauzas | gibi: ack | |
| 12:01:36 | sean-k-mooney | "Does OWNER_CYBORG trait on an RP means that _every_ RC on that RP is managed by Cyborg?" so yes traits apply to all invetories in a RP | |
| 12:01:46 | sean-k-mooney | that is why traits are on the RP not the inventory | |
| 12:02:12 | bauzas | gibi: so your comment would be about providing only traits for Cyborg VGPUs ? I'm OK if so | |
| 12:02:37 | sean-k-mooney | bauzas: we woudl have to use a forbiden trait on nova if we only do if for cyborg | |
| 12:02:38 | bauzas | the cyborg-agent could create the RPs with the same RCs | |
| 12:02:46 | gibi | "Based on these assumptions keeping the VGPU RC for nova usage and creating a new standard or custom RC for cyborg works for me. Personally I vote for a new standard trait for cyborg vGPUs." that is my summary | |
| 12:03:02 | gibi | shit | |
| 12:03:11 | gibi | Personally I vote for a new | |
| 12:03:15 | gibi | standard RC | |
| 12:03:16 | gibi | :D | |
| 12:03:31 | sean-k-mooney | so standard is where i have a problem with that approch | |
| 12:03:38 | bauzas | sean-k-mooney: not, not really | |
| 12:04:00 | sean-k-mooney | are you really suggesting that every time we want 2 service to manage the same thing that we need service speicifc resouce classes | |
| 12:05:02 | bauzas | sean-k-mooney: we could have some kind of pre-filter asking for a non-Cyborg, only if we have configuratin options for cyborg | |
| 12:05:04 | gibi | sean-k-mooney: I think we should avoid having two service manage the same resource. (you probably know that I'm against duplicating vgpu logic between nova and cyborg) | |
| 12:05:43 | sean-k-mooney | right but honestly i dont think vgpus should be in cyborg | |
| 12:05:50 | sean-k-mooney | they are not a prgramable devce | |
| 12:06:26 | bauzas | sean-k-mooney: as we discussed, it looks to me it's a non-mixed resource | |
| 12:06:41 | bauzas | if so, having another RC is understoodf | |
| 12:06:56 | gibi | sean-k-mooney: I stopped arguing on either side of moving the logic to cyborg or keeping it to nova only. | |
| 12:07:30 | gibi | sean-k-mooney: that would require one side of the table compromise | |
| 12:07:38 | gibi | sean-k-mooney: I have no bacon on either side | |
| 12:08:19 | gibi | so I don't want to force either party to accept the compromise | |
| 12:09:33 | gibi | this means we will have a sitation to manage a single type of resource from two services | |
| 12:09:38 | gibi | siutation | |
| 12:09:48 | gibi | I assume this is a rare situation | |
| 12:09:51 | sean-k-mooney | gibi: well i have been trying to get redhat to support cyborg since i was still working at intel and its not goning to happen anytime soon as in not before the A release and its not even slated for that presently | |
| 12:10:04 | sean-k-mooney | well not nessiarly | |
| 12:10:26 | bauzas | my point is to say : either we mix the same RC or not | |
| 12:10:28 | sean-k-mooney | i do think there will be other cases but in genral yes | |
| 12:10:44 | sean-k-mooney | each RC tends to be specific ot a service | |
| 12:10:44 | bauzas | if we mix, we could use the conductor to know whether we would ask cyborg or not | |
| 12:10:56 | bauzas | if we don't mix, we don't really need to have the same RC | |
| 12:10:57 | sean-k-mooney | cinder is the other example | |
| 12:11:24 | sean-k-mooney | it should be modeling its capastiy as disk_gb | |
| 12:11:36 | sean-k-mooney | which we use for local disk | |
| 12:11:43 | sean-k-mooney | ack | |
| 12:12:52 | sean-k-mooney | bauzas: gibi it sound like both of ye want to use a new resouce class so if that is what is requird to move this forward so be it | |
| 12:13:55 | swp20 | sean-k-mooney: hi, our cyborg tempest failed with the exception: Reason: 'Query' object has no attribute 'with_lockmode' . please help us with the refer:https://fc9dac502ce26d84f9de-05fc50868e17ec6a804428f62cd7e454.ssl.cf1.rackcdn.com/797427/4/check/cyborg-tempest/bd72fc3/job-output.txt | |
| 12:16:02 | sean-k-mooney | that might be form here https://github.com/openstack/cyborg/blob/ba6a35c67ee2e25cccc693d59cae2d2182aefa58/cyborg/db/sqlalchemy/api.py#L257 | |
| 12:17:16 | sean-k-mooney | swp20: i suspect that it has change in the new version fo sqlacamy | |
| 12:18:14 | sean-k-mooney | i think the excption is comming form here https://github.com/openstack/cyborg/blob/bb35be1b86953c6df5fd9a300221cd45e359e8ec/cyborg/conductor/manager.py#L150-L158 or here https://github.com/openstack/cyborg/blob/bb35be1b86953c6df5fd9a300221cd45e359e8ec/cyborg/conductor/manager.py#L170-L175 | |
| 12:18:26 | swp20 | it is because the placement changed the version of sqlalermy? | |
| 12:19:14 | swp20 | yeah, there are with_lockmode in cyborg project. | |
| 12:19:38 | sean-k-mooney | no not placment | |
| 12:19:42 | sean-k-mooney | Collecting SQLAlchemy===1.4.18 | |
| 12:20:02 | sean-k-mooney | there was a new release of sql alchemy not so log ago | |
| 12:20:15 | sean-k-mooney | and all the pojects need to adapt too it | |
| 12:21:18 | sean-k-mooney | we updated to 1.4 2 months ago https://github.com/openstack/requirements/commit/dc86260b283dedc3076d7873f5f031f45e3e3671 | |
| 12:21:33 | sean-k-mooney | it looks like that did not check for compatiablity in cyborg | |
| 12:25:35 | sean-k-mooney | "When the Query.with_lockmode() method were deprecated in favor of Query.with_for_update()..." | |
| 12:26:45 | sean-k-mooney | looks like https://docs.sqlalchemy.org/en/14/orm/query.html#sqlalchemy.orm.Query.with_for_update is the replacemnt | |
| 12:27:04 | sean-k-mooney | swp20: https://docs.sqlalchemy.org/en/14/orm/query.html#sqlalchemy.orm.Query.with_for_update is the replacment for Query.with_lockmode() | |
| 12:27:26 | swp20 | sean-k-mooney: ok, i'll try to update. thanks a lot. | |
| 12:29:31 | sean-k-mooney | swp20: it looks like it was deprecated in 1.1 or 1.2 they just finally got around to droping it in 1.4 | |
| 12:32:33 | swp20 | sean-k-mooney: thanks. the version in cyborg requirement.txt now is SQLAlchemy>=0.9.0,!=1.1.5,!=1.1.6,!=1.1.7,!=1.1.8 # MIT, i have update, hope this will success. | |
| 12:35:04 | sean-k-mooney | well you should not be capping the version locally in cyborg | |
| 12:35:39 | sean-k-mooney | that shoudl be managed via the upper constraitns file in the requirement repos | |
| 12:36:00 | sean-k-mooney | sicne all project are ment to be co installable the upper requrieemtn i manged cerntrally | |
| 12:36:12 | sean-k-mooney | so cyborg shoudl be updated to supprot 1.4 | |
| 12:36:23 | sean-k-mooney | in doing so your minium requiredm will incerease to 1.2 | |
| 12:36:40 | sean-k-mooney | for the Query.with_for_update method | |
| 12:36:58 | sean-k-mooney | actuly no | |
| 12:37:03 | sean-k-mooney | it was added in 0.9 | |
| 12:37:17 | sean-k-mooney | swp20: so you dont have to udpated your requiremetn.txt in cyborg | |
| 12:38:46 | gibi | bauzas, sean-k-mooney: yeah I think in this specific case we need to allow both service to manage vgpus and in this specific case it is a lot simpler and cleaner to have separate RCs. | |
| 12:39:25 | gibi | sean-k-mooney: for the cinder case. I think the different there is tha cinder's disk_gb is not consumable by nova but consumable by the cinder backned. And this consumability needs to be modelled | |
| 12:39:47 | sean-k-mooney | what about for generic VFs and PFs for neutron sriov | |
| 12:40:07 | gibi | good point | |
| 12:40:07 | sean-k-mooney | because there is work to support cyboprg mandaged sriov vfs also | |
| 12:40:20 | sean-k-mooney | that are assocated with a nutorn port | |
| 12:40:43 | sean-k-mooney | we can use a differet resouce class there too | |
| 12:41:05 | bauzas | again, same thoughts here | |
| 12:41:05 | sean-k-mooney | but this pattern has many implications if we repeat it | |
| 12:41:15 | bauzas | mixed case: we need to use the same RC | |
| 12:41:23 | bauzas | non-mixed case : lgtm for another RC | |
| 12:41:39 | sean-k-mooney | well its not mixed in that we alwasy iknow if its form nova or cyborg | |
| 12:41:55 | sean-k-mooney | they will have different vnic types | |
| 12:42:04 | sean-k-mooney | in this case acclerator_direct vs direct | |
| 12:42:53 | gibi | does devices allocated for accelerator_direct tracked by nova pci tracker? I think it is not it is tracked by cyborg only | |
| 12:43:06 | sean-k-mooney | its only tracked by cyborg | |
| 12:43:09 | sean-k-mooney | like the vgpus | |
| 12:43:40 | sean-k-mooney | you could whitelist and track it in the pci tracker if it was stateless | |
| 12:44:04 | sean-k-mooney | cyborg only adds the ablity to flash an image on to it if need or do other stateful things | |
| 12:45:20 | sean-k-mooney | in the case of intels current propsal they are only supportin staticly pre programed devices but plan to make it dynmaic later | |
| 12:46:04 | sean-k-mooney | the current hardware does not manage the capailtie at the VF level, its card wide hence static intially until new hardware is release that is more granular | |