Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2020-02-06
20:35:18 melwitt https://stackoverflow.com/questions/14002631/why-isnt-sqlalchemy-default-column-value-available-before-object-is-committed#14013090
20:35:20 zzzeek melwitt / mriedem not reading everyhing but when you add a column to a database that has a default value and it's "not null", the DB adds that default. that is how you get a MySQL migration that is very slow for large tables btw
20:38:02 melwitt zzzeek: is that true regardless of whether it's specified as a 'default' vs a 'server_default'? will 'server_default' also try to backfill in already existing records that do not have a value set?
20:38:16 zzzeek melwitt: oh...server default only, sorrhy
20:38:33 zzzeek melwitt: for "default" that is not a server default, I have no idea what sqlalhcemyt-migrate does
20:38:53 zzzeek i'd be surprised if they use it, though, because the "add not null column / populate server default" is necessarily atomic
20:39:00 zzzeek you can't do that from the client using a python-side default
20:39:12 melwitt zzzeek: ok, it also does a backfill, this is the patch where it's being fixed if you're curious https://review.opendev.org/706331
20:39:23 zzzeek only if migrate takes the crazy insane step of making the column as nullable first, then populating, then not-nulling
20:39:43 melwitt zzzeek: this is the link from migrate https://opendev.org/x/sqlalchemy-migrate/src/branch/master/migrate/changeset/schema.py#L594
20:40:12 zzzeek melwitt: wow, yuck :)
20:40:17 melwitt lol
20:40:20 zzzeek i hate migrate
20:41:04 zzzeek b.c. you know that fails if the DB is live and new rows still getting added
20:41:26 melwitt mnaser knows ;)
20:47:04 mnaser yeah, i do
21:15:51 sean-k-mooney Sundar: i redeploy and have been able to boot a vm with the fake cyborg driver
21:16:47 sean-k-mooney http://paste.openstack.org/show/789247/
21:16:55 efried nice
21:18:58 sean-k-mooney i will start trying different life cycle operation and testing and recording info on placement allcoation, db dumps and the like
21:19:44 sean-k-mooney i have a bash script i have written to automate this so ill script up a few test cases. and do some manually
21:20:34 sean-k-mooney since i have created a cyborg flavor i shoudl be able to run some of the standard tempest tests with that flavor
21:39:23 openstackgerrit Dan Smith proposed openstack/nova master: Fix instance.hidden migration and querying https://review.opendev.org/706331
21:39:30 lifeless stephenfin: hi, what do you need? mtreinish has commit rights on subunit
22:30:55 efried mnaser: did your +1 at PS3 here https://review.opendev.org/#/c/706331/ indicate that you had successfully tested this locally?
22:38:27 mriedem dansmith: i've got to run but will take a look at the latest later tonight
22:41:21 dansmith efried: I think he's going to have to apply it at the time he does his next upgrade which might not be for a while, it sounded like
22:44:33 melwitt dansmith: wouldn't your change unhide the instances for him today that are being incorrectly hidden? or are you saying he already fixed that via manual db update
22:44:58 dansmith melwitt: he already fixed up his db, as I understand it
22:45:05 melwitt gotcha
23:01:21 sean-k-mooney dansmith: i can check the code but are we not storing the resouce requests form the cybrog device profile in the request spec?
23:03:36 sean-k-mooney dansmith: im seeing "requested_resources": null in the request spec for the cyborg nova instance
23:04:01 sean-k-mooney the embeded flavor has "accel:device_profile": "FakeDeviceProfile"
23:05:12 sean-k-mooney however since we are not storing the groups if you change the device profil after the fact and we migration and instance or did something else that would need us to call plamcnet wwe would have to go back to cyborg which could have changed
23:18:24 dansmith sean-k-mooney: I think that's the idea
23:18:53 dansmith sean-k-mooney: you live migrate, scheduler calls placement with a new set of resources constructed from the device profile and what cyborg told you when you asked
23:19:34 dansmith sean-k-mooney: maybe we need to be doing something like examining the existing ARQs to generate those resource requests if the instance already exists?
23:19:56 sean-k-mooney new arqs sure but we dont want to hard reboot or live migate and change form an nvida gpu to an intel fpga
23:20:55 sean-k-mooney i think we need to be storing the groups retruned by cycborg when we instilly created the vm the same way we embed the flavor or image
23:20:57 dansmith not sure how that would happen on a hard reboot, but obviously agree on live-migration, but that's why I'm saying maybe we should look at the device profile on boot, and look at our existing arqs on any other move operation when asking cyborg for the resources
23:21:51 sean-k-mooney dansmith: i guess hardreboot it would not
23:22:03 sean-k-mooney we woudl just use the exising arq
23:22:10 dansmith maybe we need to ask sundar if the dp can change in a predictable or restricted way
23:22:45 dansmith I would expect the arqs can't be changed after they're allocated, and if we used those to base our new request on that would get us what we want
23:22:53 sean-k-mooney looking at the client i only see a create and delete
23:22:59 sean-k-mooney so i guess not
23:23:05 sean-k-mooney ill check the api
23:23:13 dansmith are you saying the device profile can too?
23:23:19 dansmith can change in place Imean
23:23:37 sean-k-mooney the device profile in osc can only be create or delete like a flavor
23:23:49 sean-k-mooney im going to check the api to see if there is an update
23:24:02 sean-k-mooney if they are imutable no issue
23:24:03 dansmith ah okay I thought you were implying it *could* change but you were just assuming?
23:24:07 dansmith yea
23:24:18 dansmith I need to run off for a bit, but we can sync with sundar tomorrow
23:24:30 sean-k-mooney ya no worries
23:24:46 sean-k-mooney i was just going to write a function to pull the resouce request form the nova db
23:24:59 sean-k-mooney form the request spec and compar them to the allcoations
23:25:14 sean-k-mooney to ensure they matched up
23:31:10 sean-k-mooney ok device profiels cannot be updated. https://docs.openstack.org/api-ref/accelerator/v2/index.html#device-profiles but ARQs can be whcih kind of make sense
23:31:36 sean-k-mooney you update ARQs with the host when binding or move operations
23:32:10 sean-k-mooney you cant change the device profile name however
23:37:42 openstackgerrit Dan Smith proposed openstack/nova master: Fix instance.hidden migration and querying https://review.opendev.org/706331
23:37:49 dansmith sean-k-mooney: cool
#openstack-nova - 2020-02-07
00:11:16 mriedem dansmith: a few comments in your reno
00:36:17 openstackgerrit Sundar Nadathur proposed openstack/nova master: Add Cyborg device profile groups to request spec. https://review.opendev.org/631243
00:36:18 openstackgerrit Sundar Nadathur proposed openstack/nova master: Define Cyborg ARQ binding notification event. https://review.opendev.org/692707
00:36:18 openstackgerrit Sundar Nadathur proposed openstack/nova master: Create and bind Cyborg ARQs. https://review.opendev.org/631244
00:36:19 openstackgerrit Sundar Nadathur proposed openstack/nova master: Pass accelerator requests to each virt driver from compute manager. https://review.opendev.org/698581
00:36:19 openstackgerrit Sundar Nadathur proposed openstack/nova master: Compose accelerator PCI devices into domain XML in libvirt driver. https://review.opendev.org/631245
00:36:20 openstackgerrit Sundar Nadathur proposed openstack/nova master: Delete ARQs for an instance when the instance is deleted. https://review.opendev.org/673735
00:36:20 openstackgerrit Sundar Nadathur proposed openstack/nova master: Enable hard/soft reboot with accelerators. https://review.opendev.org/697940
00:36:21 openstackgerrit Sundar Nadathur proposed openstack/nova master: Enable start/stop of instances with accelerators. https://review.opendev.org/699553
00:36:21 openstackgerrit Sundar Nadathur proposed openstack/nova master: Enable and use COMPUTE_ACCELERATORS trait. https://review.opendev.org/699554
00:36:22 openstackgerrit Sundar Nadathur proposed openstack/nova master: Bump compute rpcapi version and reduce Cyborg calls. https://review.opendev.org/704227
00:36:22 openstackgerrit Sundar Nadathur proposed openstack/nova master: Add cyborg tempest job. https://review.opendev.org/670999
08:29:01 gibi efried: ack, I will try to spend some time on the cyborg series today
10:22:01 stephenfin gibi, lyarwood: real quick, do either of you see anything obviously wrong in https://zuul.opendev.org/t/openstack/build/5b196c5a7cf944df8857a08dc15aa79f/console ?
10:22:15 stephenfin It's saying "TypeError: create_port_binding() missing 2 required positional arguments: 'port_id' and 'data'"
10:22:31 stephenfin But line it's failing on is obviously passing those?
10:22:32 stephenfin "binding = client.create_port_binding(port_id, data)['binding']"
10:23:29 stephenfin gibi: Also, low priority but I updated a commit message on a patch you'd previously approved, if you have time to revisit https://review.opendev.org/705655
10:24:00 gibi stephenfin: could you link the patch that produced such test result?
10:24:09 stephenfin gibi: https://review.opendev.org/#/c/706295/
10:25:30 gibi interesting :)
10:25:38 stephenfin right?
10:25:40 gibi pulling it down
10:29:02 gibi trying to reproduce locally..
10:34:06 gibi stephenfin: nova.tests.fixtures.NeutronFixture.get_port_binding has get_port_binding(self, context, client, port_id, host) signature so when the code pass two positional args it fills only client and context and not the port and host
10:34:18 stephenfin gdi
10:34:40 stephenfin gibi++ Thanks /o\ :)
10:34:44 gibi no problem
10:34:48 stephenfin I'd been staring at that for an hour
10:34:50 gibi the error message was deeply missleading
10:35:01 stephenfin Yuuup. Bad coincidence :)
10:35:27 gibi yepp
10:50:40 openstackgerrit Stephen Finucane proposed openstack/nova master: Remove universal wheel configuration https://review.opendev.org/706466
11:02:26 openstackgerrit Brin Zhang proposed openstack/nova master: Add new default roles in os-instance-actions policies https://review.opendev.org/706470
11:05:12 lyarwood stephenfin: sorry missed that, still need me to take a look?
11:05:25 stephenfin lyarwood: Nope, we sussed it

Earlier   Later