| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-03-20 | |||
| 21:39:55 | mriedem | dansmith: i don't disagree, it would require thought, | |
| 21:40:13 | mriedem | so i think the answer to klindgren's question is it's going to require thought, not just a simple single change | |
| 21:40:18 | klindgren | I just want to avoid a patch that we can never get merged upstream | |
| 21:40:34 | efried | mriedem: yes. or create a new blank patch. | |
| 21:40:34 | mriedem | i suspect there would be MigrateData object RPC changes and such to communicate between the dest and source compute | |
| 21:40:56 | dansmith | mriedem: agree it's non trivial | |
| 21:41:27 | mriedem | klindgren: are you or your coworker buddies going to be at the ptg? | |
| 21:41:34 | efried | oo, cdent do tell | |
| 21:41:52 | klindgren | mriedem, yes we will have 2 people there | |
| 21:41:53 | mriedem | cdent: did someone put potato skins down the drain again? | |
| 21:41:55 | klindgren | I won't be there | |
| 21:42:03 | cdent | mriedem: twice | |
| 21:42:03 | klindgren | afaik | |
| 21:42:08 | mriedem | klindgren: at least throw something on here https://etherpad.openstack.org/p/nova-ptg-train | |
| 21:42:23 | cdent | efried: I'm relatively certain the same config is being used twice and it should be. need to confirm | |
| 21:42:28 | mriedem | and if you start coming up with design ideas and have thought through the corner cases and such, it'd be good to dump those thoughts in the ML | |
| 21:42:33 | dansmith | mriedem: injection_info = InjectionInfo(network_info=network_info, | |
| 21:42:33 | dansmith | files=injected_files, | |
| 21:42:44 | cdent | and I'm not clear why this only matters for logging opts | |
| 21:42:48 | cdent | but we'll see | |
| 21:42:52 | dansmith | mriedem: ^ is where we pass in the injected files to be written to the configdrive | |
| 21:43:22 | dansmith | mriedem: so I don't think host config disabling injection tells us anything about what is in the configdrive image | |
| 21:43:34 | mriedem | ok | |
| 21:43:38 | mriedem | RequestSpec to the rescue! | |
| 21:43:52 | dansmith | or system_metadata, a count of injected files we assert to be nonzero or something | |
| 21:44:00 | dansmith | but still, it's going to be confusing for admins, | |
| 21:44:15 | dansmith | wondering why some instances are migratable and/or don't depend on ssh keys and some do | |
| 21:44:18 | dansmith | but..yeah. | |
| 21:45:15 | dansmith | if we had the http service integrated into compute for the health check, we could do something better, like allowing one compute to fetch the image via one-time token from the source | |
| 21:46:11 | dansmith | i.e. fetch http://sourcehost/migration_assets/$uuid | |
| 21:46:18 | dansmith | they have to be able to talk to each other anyway | |
| 21:46:28 | dansmith | and just make that self-destruct as soon as the GET is performed | |
| 21:47:44 | mriedem | have you changed topics? or in the wrong channel perhaps? :) | |
| 21:47:50 | dansmith | no? | |
| 21:47:53 | mriedem | oh | |
| 21:47:55 | mriedem | heh | |
| 21:48:09 | mriedem | i'm not sure where "if we had the http service integrated into compute for the health check" came from | |
| 21:48:32 | dansmith | thinking of better ways to move bulk data between compute nodes than requiring ssh keys and allowing logins | |
| 21:48:44 | dansmith | because that _is_ totally nuts from a security perspective, | |
| 21:49:06 | dansmith | and fixing that with something more secure would be better than just adding a different path (regen) which only works in some cases anyway | |
| 21:49:55 | dansmith | such an approach would work for and fix the security holes in cold migration as well | |
| 21:50:26 | mriedem | well, it's essentially what i'm doing for cross-cell resize with glance | |
| 21:50:40 | dansmith | using glance as the intermediary, but yeah | |
| 21:50:59 | mriedem | noting now that your config drive will have to get rebuilt on the other side... | |
| 21:51:11 | mriedem | does config driver already get rebuilt on resize? | |
| 21:51:15 | mriedem | *drive | |
| 21:51:20 | dansmith | I thought it gets copied | |
| 21:51:34 | dansmith | along with the root disk | |
| 21:51:57 | mriedem | i'm not sure https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L8882 | |
| 21:52:00 | dansmith | if we had this http endpoint, your cross-cell resize stuff could be much more like regular resize and not have to go through glance | |
| 21:52:34 | dansmith | mriedem: that doesn't run if it already exists | |
| 21:52:44 | dansmith | so yeah, maybe that means it's regen'd but if it's copied then it won't | |
| 21:53:03 | dansmith | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L3626 | |
| 21:54:17 | mriedem | hmm yeah so i suppose disk.config gets copied here https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L8811 | |
| 21:55:39 | mriedem | huh, i guess that's something that needs to be at least documented for the cross-cell stuff as a limitation right now because it's not doing anything with the config drive, so it's going to get regenerated on the dest and you'd lose any personality files | |
| 21:56:40 | dansmith | you're welcome. | |
| 21:58:23 | mriedem | duly noted in my re-proposed spec https://review.openstack.org/#/c/642807/1 | |
| 21:58:41 | mriedem | i don't know what the answer is, upload a snapshot of the disk.config to glance? | |
| 22:00:04 | dansmith | no the answer is implement an http server :D | |
| 22:03:38 | cdent | efried, mriedem : the gist of the problem is that you can register other opts multiple times, but not cli opts, and adding oslo_log.log.register_opts() to the common path brings CLI options into the scene of test_local_delete_removes_allocations_after_compute_restart, where config is registered multiple times (3 in fact) | |
| 22:04:11 | cdent | it only shows up in that test because we are doing some wildness with the PlacementFixture that doesn't happen anywhere else | |
| 22:04:27 | cdent | candidates for fixing it: | |
| 22:04:36 | cdent | 1. that revert, worry about it later | |
| 22:05:24 | cdent | 2. change that test to do things differently (hard because the reference to the in ram sqlite database needs to be maintained) | |
| 22:05:53 | cdent | 3. change the placement fixture (on the placement side) to take a flag to register or not register opts | |
| 22:06:17 | cdent | 3 is probably the best choice in the long run, and won't take me long to do | |
| 22:06:27 | efried | why does the placement fixture need to register opts ever? | |
| 22:06:45 | cdent | because it has to use opts in order to start placement | |
| 22:06:49 | efried | I guess when tests go after opts, the defaults need to be there. | |
| 22:06:51 | efried | and that | |
| 22:07:10 | cdent | unfortunately the way oslo_config works, it has to know about stuff | |
| 22:07:15 | cdent | (good and bads to that) | |
| 22:07:28 | efried | 3 today, or should we 1 to unblock and then 3? | |
| 22:07:54 | cdent | the mode the fixture has now is "don't make the caller think, if possible" | |
| 22:08:06 | cdent | I'm happy with just 3 if everyone else is | |
| 22:08:19 | cdent | but if you want to unblock quicker, then 1 and then 3 | |
| 22:08:40 | efried | yeah, let's merge 1 and you can put 3 on top and include unrevert in it? | |
| 22:09:13 | cdent | I wanted to be able to find a version of 2, but that test is already so hairy I don't think it is worth | |
| 22:09:14 | cdent | sure | |
| 22:09:33 | cdent | feel free to push the revert through | |
| 22:10:03 | efried | cdent, mriedem, melwitt: fast-approved https://review.openstack.org/#/c/645014/ | |
| 22:10:44 | efried | and then mriedem respun and did same, neat | |
| 22:10:51 | mriedem | i added the bug tag | |
| 22:11:06 | efried | yup, ++ | |
| 22:12:58 | openstackgerrit | Eric Fried proposed openstack/nova master: DNM: Trying a different strategy for tests https://review.openstack.org/645026 | |
| 22:19:57 | melwitt | efried: ack, thanks | |
| 22:25:45 | mriedem | well now i'm depressed and have a 2 hour call in 3 hours so it's time to leave | |
| 22:35:40 | NobodyCam | anyone have any pointers to stacking down instances stuck in scheduling .. I set nova* to debug mode and don't see any errors jumping out | |
| 22:36:26 | NobodyCam | s/stracking/tracking | |
| 22:36:39 | openstackgerrit | Chris Dent proposed openstack/nova master: Don't register placement opts mutiple times in a test https://review.openstack.org/645033 | |
| 22:37:37 | cdent | mriedem_away, efried ^ that fixes the nova side and depends-on the placement side fix | |
| 22:39:02 | NobodyCam | I do see this in the compute log: | |
| 22:39:04 | NobodyCam | Instance 7d8f61e6-155c-47a9-9d35-95939b29d487 has been scheduled to this compute host, the scheduler has made an allocation against this compute node but the instance has yet to start. Skipping heal of allocation: {u'resources': {u'MEMORY_MB': 1024, u'VCPU': 1, u'DISK_GB': 20}}. _remove_deleted_instances_allocations /openstack/venvs/nova-18.1.5.dev5/lib/python2.7/site-packages/nova/compute/resource_tracker.py:1355 | |
| 23:16:22 | aspiers | cdent: OK, now you are not just ahead of me but downright telepathic. I was about to report that placement issue... | |
| 23:17:00 | cdent | aspiers: which one? | |
| 23:17:17 | aspiers | the option registering | |
| 23:17:38 | cdent | that's mriedem_away's psychic powers not mine | |
| 23:18:49 | aspiers | OK, so you just fixed it :) | |
| 23:19:12 | aspiers | Either way it's nice when things get fixed even before I get a chance to report them | |
| 23:19:17 | aspiers | haha | |