Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-02-07
19:43:48 sean-k-mooney yes im saying we didnt want to do a microversion bump for evey one
19:47:18 sean-k-mooney speaking of op i just booted a vm with your latest revision so ill go test some of them
19:48:23 Sundar Sure. Thanks.
19:50:08 sean-k-mooney my isp broke my home network so its takeing longer then i hoped to test this. i might set up the multi node setup on my laptop instead of my home openstack on monday but ill do what i can on a singel node first
20:09:13 efried Sundar: I've always been in favor of that idea (blocking unsupported operations with a useful message rather than letting them fail "organically" and mysteriously), but I know others disagree.
20:11:25 sean-k-mooney for what its worth i have just done boot,stop,start,reboot,add/remove volume, rescue/unrescue and so far no errors
20:11:56 sean-k-mooney i am not seeing any real interaction with cyborg during those operations
20:12:42 sean-k-mooney which is more or less expected but im not sure if we would loss the acclerator when we regenerate the xml
20:13:26 sean-k-mooney i will look at this more closely next week but right now im just check to make sure the operation complette correctly
20:29:01 Sundar sean-k-mooney: Great. FWIW, I do most of these with FPGAs. Except rescue/unrescue with different images. The list of ops i have checked are in: https://review.opendev.org/673735
20:29:44 sean-k-mooney im still makeing my way through the list.
20:30:11 sean-k-mooney im hoping to get access to a real server with a rush creek or vista creek next week or the week after
20:30:20 sean-k-mooney which one are you using again?
20:31:09 sean-k-mooney i have jsut done add/remove network interface, pause,unpause, suspend,resume, lock,unlock and rebuild
20:31:13 Sundar Rush Creek, DCP 1.2
20:31:40 sean-k-mooney cool we have 1 server i think with one of each so ill ask for the rush creek system
20:31:56 Sundar OPAE version 1.1.2-1
20:32:26 Sundar The OPAE packages come with some sample bitstreams, sp. NLB modes 0 and 3. I use both.
20:33:58 Sundar I can also be reached at ns1.sundar AT gmail DOT com if there is a need for more detailed responses or file transfers.
20:35:29 Sundar I have some utility functions to create/delete device profiles, ARQs using curl. ould be happy to share them if you prefer.
20:37:01 sean-k-mooney im just using the openstack client and i have a test script
20:38:26 Sundar The openstack client is WIP. There are patches to recast it to use openstacksdk etc. https://review.opendev.org/#/c/681391/
20:38:48 sean-k-mooney yep im using that
20:39:02 sean-k-mooney with the openstacksdk patch too
20:39:26 sean-k-mooney there are a bunch of design issue with it that should be adressed but its kind of useable
20:40:11 sean-k-mooney like when you create a device profile you can only spify its name not the uuid but it only allows you to show a device profile by uuid not name
20:40:53 Sundar Yea, agreed. The APIs allow both name and uuid though.
20:41:12 sean-k-mooney yep so the patches are just incomplete
20:41:22 sean-k-mooney i have worked around that
20:42:08 Sundar Do you have any pointers for me to look at the rescheduling question?
20:43:37 sean-k-mooney i have not read the question so no. i was going to try and force it by making the vm spawn fail
20:44:29 sean-k-mooney ok so first bug. when i shelve the instance the arq is still bound to the host when the vm is shevle_offloaded
20:45:01 Sundar Yup, I have not added support for shelve. The Delete ARQ patch states what I support.
20:45:25 sean-k-mooney yep but we want to check them all anyway
20:46:01 Sundar Basically, I prioritized the basic ops. I asked some folks outside Intel what they do with FPGAs in their lab with Cyborg, and went with that.
20:47:56 sean-k-mooney ya which is fine. unshevel "works" but i dont see any interacation with cyborg os i suspect it would not have the accellerator attached after unshelve.
20:48:08 sean-k-mooney again thats fine we just need to document it
20:49:32 Sundar ok
20:50:25 sean-k-mooney ok so after unshelve the allocations do not container the fake device so ya that means the device woudl be lsot
20:54:14 sean-k-mooney it looks like resize to a different flavor and then back does not fix the placement allcoation
20:54:20 sean-k-mooney it did however complete
20:58:49 Sundar Yes, I have only tested with resize to the same flavor, which is of course a no op. I was just making sure that there is no basic gotcha.
20:59:30 sean-k-mooney this is what i tested with the fake driver today
20:59:32 sean-k-mooney http://paste.openstack.org/show/789306/
20:59:56 sean-k-mooney Sundar: resize to same flavor should be blocked in the api
21:00:00 sean-k-mooney it is in the client
21:00:13 sean-k-mooney you can migrate but resize to same flavor is invalid
21:01:22 Sundar remove-vol/net: unrelated to Cyborg, right?
21:01:50 sean-k-mooney yes but you blocked them in your patch that check for the flavor extra spec
21:01:56 sean-k-mooney they seam to work fine
21:02:05 Sundar Suspend will not work with real FPGAs, because libvirt will error out with: Domain has assigned non-USB devices.
21:02:07 sean-k-mooney at least with the fake dirver
21:02:38 sean-k-mooney suspend shoudl do a managed save which will detach all hostdev devices
21:02:46 Sundar delete failed?
21:02:49 sean-k-mooney yes
21:03:12 sean-k-mooney it failed in the last versions too
21:03:34 sean-k-mooney the conductor explodes with an unexpected respoce form cyborg
21:03:48 sean-k-mooney when it tries to delete /unbind the arqs
21:04:03 Sundar Hmm, please send me the logs. I do deletes all the time, but your sequence of ops probably triggered something.
21:04:39 sean-k-mooney i found that was broken just with boot then delete two days ago
21:04:56 sean-k-mooney ill need to unstack and stack to be able to test it again
21:05:18 Sundar I'll try boot + delete. Was it hard reboot?
21:06:36 sean-k-mooney no just boot then delete
21:06:47 sean-k-mooney hardreboot seams to be fine
21:08:23 sean-k-mooney Sundar: https://etherpad.openstack.org/p/sean-cyborg-testing-delete-logs
21:08:36 sean-k-mooney paste.openstack.org isnt loadign for me
21:08:44 sean-k-mooney but that is all the nova and cyborg logs
21:10:19 sean-k-mooney it look like the cyborg api returned a 401
21:12:19 Sundar devstack@cyborg-api.service[26903]: 2020-02-07 20:56:05.719 .... Authorization failed for token: keystonemiddleware.auth_token._exceptions.InvalidToken: Token authorization failed.
21:12:53 sean-k-mooney yep
21:13:35 sean-k-mooney so the call to cyborg either need an admin token which i dont think is correct or the token expired and you need to handel that
21:14:29 sean-k-mooney deleting the vm again does not fix it by they way
21:14:40 sean-k-mooney but i can create and delete non cyborg vms
21:15:05 sean-k-mooney i get the same 401 reponce on the second attempt
21:15:13 sean-k-mooney but i can delete hte arq myself
21:15:58 Sundar What does it take to handle an expired token? BTW: https://opendev.org/openstack/cyborg/src/branch/master/cyborg/common/policy.py#L86 applies defuault rule for ARQ deletes
21:16:39 Sundar *default rule: rule:admin_or_owner
21:17:02 sean-k-mooney so there is basically two ways to try and resovle it or you can raise an error
21:17:38 sean-k-mooney if service tokens are configured youc an fall back to using those alternitvly you can have an admin token for the servcie in the nova conf and escalate
21:17:42 sean-k-mooney or you can fail
21:17:51 sean-k-mooney but in this can my token was not expired
21:18:14 sean-k-mooney when i did it again i was issue a new token and it failed again
21:18:58 sean-k-mooney my devstack also issue tokesn that last for an hour so it shoudl be fine
21:20:41 sean-k-mooney anyway im done for the day o/
21:21:21 Sundar Ok. Not sure what is going wrong here. I'll try it out as non-admin.
21:21:34 Sundar Thanks a lot for all your efforts. ;) Have a good weekend!
21:43:38 Sundar efried: Re. "blocking unsupported operations", the objection seems to be that each change requires microversion changes.
21:45:49 Sundar FWIW, I think any further changes after this series will be few and far between. We could put in a microversion change now. Any further changes should be clumped together as much as possible, based on operator requirements.
#openstack-nova - 2020-02-08
03:02:28 openstackgerrit HYSong proposed openstack/nova master: fix bug 1861964 https://review.opendev.org/706647
03:02:28 openstack bug 1861964 in OpenStack Compute (nova) "availability_zone in request_specs not updated when live migrate vm from one availability zone to another" [Undecided,New] https://launchpad.net/bugs/1861964
05:15:09 openstackgerrit HYSong proposed openstack/nova master: fix format https://review.opendev.org/706655
16:16:26 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Add test coverage of existing admin_password policies https://review.opendev.org/701629
16:17:09 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Add test coverage of existing admin_password policies https://review.opendev.org/701629
16:17:17 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Introduce scope_types in os-admin-password https://review.opendev.org/701630
16:17:32 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Add new default roles in os-admin-password policies https://review.opendev.org/701639
16:20:05 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Add new default roles in os-agents policies https://review.opendev.org/701648
17:02:18 openstackgerrit Merged openstack/nova master: Fix instance.hidden migration and querying https://review.opendev.org/706331
18:24:10 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Add new default roles in os-agents policies https://review.opendev.org/701648
18:24:26 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Pass the actual target in os-agents policy https://review.opendev.org/701649
18:37:36 openstackgerrit Ghanshyam Mann proposed openstack/nova master: Add new default roles in os-atttach-inerfaces policies https://review.opendev.org/706672

Earlier   Later