Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2023-10-16
16:35:08 sean-k-mooney ah i assume your watching the live output
16:35:55 dansmith yup
16:36:09 dansmith just passed the main job as well
16:39:02 sean-k-mooney added link to the passing job where it ran and left a +2
16:39:48 sean-k-mooney if we start seeing this type of dep issue often i woudl prefer to use bindep or similar to store the packages in a file or move the list of packagges to a role default
16:39:54 sean-k-mooney but for now this is fine
16:41:42 dansmith ack, well, I'd like to get the gate unblocked at least
16:42:26 dansmith I know bindep is what we use for that stuff normally, but I figured it was more aimed at setup for running the services and not just for ansible stuff to work
16:42:34 dansmith I suppose it's also usable
16:42:45 sean-k-mooney bindep has tags
16:42:51 sean-k-mooney so we can use [ansible]
16:43:02 sean-k-mooney for example if we wanted to store ansible only deps or whatever
16:43:22 dansmith ack
16:43:44 sean-k-mooney we do use the "test" tag for functional/unit tests deps
16:43:47 dansmith so maybe I should just slap this into bindep.txt instead of doing this in the ansible?
16:43:59 dansmith or maybe just follow this up with that so we can unblock the gate
16:44:20 sean-k-mooney well bindep is likely not aviable on the host outside of the venv where devstack installed it
16:44:33 sean-k-mooney so if we want to go that way lets do it in a followup
16:44:47 sean-k-mooney as first we need to decied how to make bindep avaialable
16:45:30 dansmith oh, we install bindep in the venv to manage deps on the host? I figured that was installed and managed before we got to devstack at all
16:45:46 dansmith yeah, let me queue up something behind this if for no other reason than to serve as a reminder
16:46:05 sean-k-mooney for the devstack job i belive its in the venv for the unit test jobs its installed in the job
16:46:09 sean-k-mooney before we invoke tox
16:46:45 sean-k-mooney https://github.com/openstack/nova/blob/9dceb735f6926c71a8c2e2c6ef8b5e0a1363041d/bindep.txt#L24
16:46:58 sean-k-mooney lxml is a dep in bindep already for what its worth
16:47:04 sean-k-mooney but its marked as a test dep
16:47:21 sean-k-mooney so its not installed by default by devstack
16:47:45 sean-k-mooney actully no
16:47:50 sean-k-mooney thats not the python bindings
16:48:01 sean-k-mooney that the clib that lxml uses
16:48:09 dansmith yeah, neither is libvirt-python, but that's clearly installed which is weird that it works
16:48:17 dansmith er, python3-libvirt
16:48:52 sean-k-mooney i belive that will be install by devstack seperatly
16:49:15 sean-k-mooney https://github.com/openstack/devstack/blob/ca4d5132e63752878620c4e4f374d98d433b3f52/lib/nova_plugins/functions-libvirt#L96
16:49:42 sean-k-mooney and here for ubuntu https://github.com/openstack/devstack/blob/ca4d5132e63752878620c4e4f374d98d433b3f52/lib/nova_plugins/functions-libvirt#L72C87-L72C102
16:49:49 opendevreview Dan Smith proposed openstack/nova master: Move lxml dep to bindep.txt https://review.opendev.org/c/openstack/nova/+/898439
16:50:14 dansmith sean-k-mooney: right but I would expect python3-libvirt to have a hard dep on python3-lxml, but perhaps not
16:50:35 sean-k-mooney i dont think it does
16:50:41 sean-k-mooney we need lxml to parse the xml
16:50:49 sean-k-mooney but the xml is generated by libvirt
16:50:54 sean-k-mooney which would use the c lib
16:50:59 sean-k-mooney not the python bindings
16:51:49 dansmith I guess I expected some of the python bindings to manipulate xml chunks themselves, but perhaps not
16:52:46 sean-k-mooney so just confirmign that his would work devstack invokes bindep if teh bindep file exits for a project here in setup_package https://github.com/openstack/devstack/blob/ca4d5132e63752878620c4e4f374d98d433b3f52/inc/python#L465
16:55:49 sean-k-mooney that is called form _setup_package_with_constraints_edit which is called form setup_deploy here https://github.com/openstack/devstack/blob/ca4d5132e63752878620c4e4f374d98d433b3f52/inc/python#L359
16:56:08 sean-k-mooney and we call that for nova here https://github.com/openstack/devstack/blob/ca4d5132e63752878620c4e4f374d98d433b3f52/lib/nova#L971
16:56:25 sean-k-mooney so based on that call graph
16:56:31 sean-k-mooney we are not specifying any profile
16:56:54 sean-k-mooney i.e. it sthe same as just runing bindep in the nova root effectivly
16:57:26 sean-k-mooney so we just need to confirm that https://review.opendev.org/c/openstack/nova/+/898439 make it included in that properly
16:57:43 sean-k-mooney ah you added no tag so yes it will be
17:36:26 dansmith bauzas: can you +W this to unblock the gate please? https://review.opendev.org/c/openstack/nova/+/898435
17:36:45 bauzas the quickiest I was able to
17:36:55 dansmith thanks
17:38:47 dansmith sean-k-mooney: the bindep approach doesn't leave us with lxml installed
17:41:30 sean-k-mooney ack i wonder if we are removing it in devstack to aovid pip issues
17:42:41 sean-k-mooney id dont see anything that would remove it but lets stick with teh simple solution for now
17:42:56 dansmith idk, but glad we approved the first solution in the short term :)
17:51:04 sean-k-mooney dansmith: for what its worth i dont see any packages being installed
17:51:17 dansmith sean-k-mooney: bindep packages you mean?
17:51:30 dansmith I didn't see it try lxml that's for sure
17:51:48 sean-k-mooney ya bindep is installed bu ti dont see ay execution of setup_package that actully installs anything
17:52:16 sean-k-mooney so that might have silently been broken at some point
17:52:38 sean-k-mooney im sing things like inc/python:_setup_package_with_constraints_edit:387 : [[ /opt/stack/neutron == -bindep* ]]
17:53:06 dansmith I'd have expected more breakage in that case
17:54:01 sean-k-mooney ah so reading the code
17:54:08 sean-k-mooney https://github.com/openstack/devstack/blob/master/inc/python#L385
17:54:20 sean-k-mooney the fintion is checkign if it was invoked with -bindep
17:54:23 sean-k-mooney as a flag
17:54:33 sean-k-mooney and poping it
17:54:39 sean-k-mooney and we are not doign that
17:55:04 sean-k-mooney we do not pass -bindep when seting up nova
17:55:22 dansmith probably need to get gmann to consult here I think
17:55:35 sean-k-mooney https://github.com/openstack/devstack/blob/ca4d5132e63752878620c4e4f374d98d433b3f52/lib/nova#L971
17:55:53 sean-k-mooney i dont see any usage of this in tree so maybe this was added for out of tree plugins to call
17:56:22 sean-k-mooney https://github.com/openstack/devstack/commit/58243f6203f3385039fe7124b037d5abf7e1d59e
17:58:59 sean-k-mooney we can always follow up in #openstack-qa but my read on this is the infra is in place but not hooked up to be called
17:59:34 sean-k-mooney this seams to be the only? usage https://github.com/openstack/devstack/commit/dd3731c86a13334125715e44880491042526227c
18:00:06 sean-k-mooney so it be used if we use libs_from_git=nova i guess but that shoudl be the case in teh ci job
18:01:39 sean-k-mooney am not quite its still only takign that path for specific lib
18:02:01 sean-k-mooney not anything listed in libs form git so ya it is not hooked up for nova/service proejcts
18:02:49 sean-k-mooney it shoudl be simple enough to hook up but we shoudl check with gmann et al first to see if there is a reason to only do this for libs
18:03:09 dansmith yeah seems like there must be something missing
18:05:31 sean-k-mooney i think to really do this properly and get the most use out of it we woudl want to modify lib/nova and the nova hypervisor plugsin to pass a diffetn tag (the virt driver) and on the nova side group driver deps by tag .e.g libvirt or ironic in the bindeps file
18:06:03 sean-k-mooney it would be a nice enhancement but a lot of work to fully get right.
18:06:09 sean-k-mooney im honestly fine with your first patch
18:06:51 sean-k-mooney i would only look at hooking up bindep in devstack if we find ourselve needing to do this more hten once or twice a cycle
18:08:32 gmann dansmith: sean-k-mooney hi
18:08:46 sean-k-mooney o/
18:09:15 gmann bindeps can be installed two way, via flag while installing nova or have file in devstack/file/bindep.txt itself which will be indalled by default
18:09:19 gmann installed
18:09:54 gmann but nova needs to have that in-tree under devstack directory with plugin setup
18:10:13 sean-k-mooney gmann: yep so in this case its a dep for an ansible module we are using in one specific ci job post playbook
18:10:15 gmann I mean here - https://github.com/openstack/nova/tree/master/devstack
18:10:28 sean-k-mooney the dep also happens to be a dep nova has
18:11:01 sean-k-mooney we are assumign that until last week lxml was insallsed transitivly via a weak dep somewhere
18:11:07 sean-k-mooney so the ci playbook worked
18:11:18 sean-k-mooney and that weak dep got lost over the weekend
18:11:43 gmann ohk so we want to have explicit in nova bindep right?
18:12:21 sean-k-mooney well for now we jsut installed it in the job. but then i mentioned bindep to dansmith but that dint work becasue we dont install nova's bindep currently

Earlier   Later