| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-06-16 | |||
| 19:30:22 | mriedem | i believe harlowja was also talking about this in an operators ML thread at one point | |
| 19:30:30 | harlowja | wut | |
| 19:30:36 | NostawRm | it wouldn't be the first time that I thought something would be useful and there's good reason not to do it lol | |
| 19:30:47 | mriedem | harlowja: refresh config drive on reboot | |
| 19:30:51 | harlowja | ah | |
| 19:31:03 | mriedem | this thread http://lists.openstack.org/pipermail/openstack-operators/2017-May/013548.html | |
| 19:31:07 | harlowja | ya, so its i mean theoritically possible | |
| 19:31:19 | harlowja | i just know that cloud-init won't refresh/reread that data | |
| 19:31:29 | harlowja | (so normal images using cloud-init won't understand such a refresh) | |
| 19:31:42 | harlowja | it only will re-read things if the instance-id changes | |
| 19:31:51 | harlowja | or u delete the cloud-init data cache lol | |
| 19:32:04 | NostawRm | oh, it won't? I didn't know that about cloud-init | |
| 19:32:10 | harlowja | ya | |
| 19:32:29 | NostawRm | bah, caches, the bane of my day | |
| 19:32:49 | harlowja | example, https://gist.github.com/harlowja/e66514030588d7fcebcaa152ce7ec0be | |
| 19:32:55 | harlowja | its cache | |
| 19:34:06 | harlowja | u can wipe `/var/lib/cloud/instances/` and that will cause it to refresh, but its not designed to understand such refresh (unless instance-id gets transformed) | |
| 19:34:45 | NostawRm | yeah, and that's no good, which means that even going the route of using DHCP again for the metadata service isn't going to help | |
| 19:34:55 | harlowja | yup | |
| 19:35:19 | harlowja | the link mriedem showed was about how there are already mutable things in the metadata service (that my guess most people don't know about lol) | |
| 19:36:00 | harlowja | and those mutable things aren't propagated to the config-drive | |
| 19:36:14 | harlowja | so there is a bug/feature-discrepancy/hidden-feature? already | |
| 19:36:35 | NostawRm | I think that puts me building a per-boot script that pulls what I want :/ | |
| 19:36:47 | harlowja | ya, just be careful imho | |
| 19:36:52 | harlowja | we have a cron job that does something like that | |
| 19:37:02 | harlowja | and it can create a bunch more traffic than u would normally get :-P | |
| 19:37:12 | harlowja | (on the metadata service) | |
| 19:37:26 | harlowja | so use it with caution | |
| 19:37:42 | cfriesen | dansmith: mriedem: anyone else: why did we keep db.pcidevice_online_data_migration and db.aggregate_uuids_online_data_migration in Newton online data migrations when they were already in Mitaka? | |
| 19:37:42 | harlowja | espcially if u put it in all vms... | |
| 19:38:14 | NostawRm | I'd hope it wouldn't be too bad with it being per boot and not a timed job, but I'm probably underestimating | |
| 19:38:40 | harlowja | NostawRm just do some calculations | |
| 19:38:45 | harlowja | just incase :-P | |
| 19:39:11 | mriedem | figleaf: i took the liberty https://review.openstack.org/#/c/473627 | |
| 19:40:06 | mriedem | cfriesen: because you can install the new code in a venv or container and run the migrations before upgrading from mitaka to newton | |
| 19:40:07 | NostawRm | I genuinely appreciate the input harlowja, I kept wanting to keep everything contained to the config_drive and cloud-init because it felt 'proper' but it is what it is | |
| 19:40:41 | mriedem | cfriesen: also, it's not science, sometimes we just don't cleanup for a release or two | |
| 19:40:41 | mriedem | since it's not urgent | |
| 19:40:50 | harlowja | NostawRm ya | |
| 19:40:52 | mriedem | cfriesen: e.g. https://review.openstack.org/#/c/460377/ | |
| 19:41:10 | cfriesen | NostawRm: config drive is tricky, it's not persisted in the DB anywhere. | |
| 19:41:13 | figleaf | mriedem: ok, I can log it. If operators shoot themselves in the foot that way... | |
| 19:42:03 | NostawRm | cfriesen, yeah, I've noticed, I've been digging which is why I felt okay poking about it in the dev channel :P | |
| 19:42:19 | NostawRm | meta data in general seems finicky, or atleast in my use case | |
| 19:42:23 | cfriesen | NostawRm: I have used the mutable metadata service stuff though | |
| 19:44:02 | NostawRm | like I'm going to be giving multiple IPs to the interfaces of instances, ant the metadata service just picks the first IP but that may not be the 'main ip' that was set aside for licensing purposes :/ | |
| 19:44:45 | harlowja | cfriesen NostawRm ya, i think smoser (another cloud-init guy, the main guy really) doesn't want config-drive at all, lol | |
| 19:44:51 | cfriesen | mriedem: doing an upgrade from Mitaka to Newton, one of the "type-PF" entries in table pci_devices ended up with a non-null entry under the "parent_addr" column, which is I think unexpected. Not sure yet if it's upstream code or ours, but its odd. | |
| 19:45:33 | mriedem | cfriesen: i believe that's already fixed | |
| 19:45:52 | NostawRm | harlowja, :/ that's odd, I like the config-drive better, partially because I wasn't going to use DHCP | |
| 19:45:59 | mriedem | cfriesen: https://github.com/openstack/nova/commit/ad12fa65f5cc06bdee52be49b7370d703855d618 | |
| 19:46:06 | cfriesen | harlowja: I proposed dropping it and allowing specifying personality files on an evacuate the way we do now for rebuild. (since config drive gets lost on evacuate currently) | |
| 19:46:31 | cfriesen | NostawRm: by the way, config drive gets lost on evacuate. :) | |
| 19:46:34 | harlowja | cfriesen right | |
| 19:46:34 | mriedem | cfriesen: you should be getting the latest stable branch releases before doing your upgrade | |
| 19:46:49 | NostawRm | cfriesen, don't think they do when using rbd, but I'd have to double check | |
| 19:47:13 | harlowja | NostawRm ya, i mean i don't mind config-drive, its just more flakey than a webservice | |
| 19:47:17 | mriedem | eandersson: superdan: mitaka backport https://review.openstack.org/475067 | |
| 19:47:21 | harlowja | especially i think in migrations, live-migrtions.... | |
| 19:47:41 | harlowja | some of those were prob bugs that got fixed, but its always seem to have been more 'finiky?' | |
| 19:48:42 | superdan | mriedem: thanks for doing that | |
| 19:49:36 | mriedem | superdan: here is that expected nova-status upgrade check warning for the CoreFilter thing from bauzas' change http://logs.openstack.org/00/427200/4/check/gate-grenade-dsvm-neutron-ubuntu-xenial/59e0c78/logs/grenade.sh.txt.gz#_2017-06-16_17_32_25_590 | |
| 19:49:40 | mriedem | https://review.openstack.org/#/c/427200/ | |
| 19:49:49 | mriedem | in grenade we ignore everything below failure | |
| 19:50:40 | mriedem | i figure if/when we drop ram/disk filters from enabled_filters defaults, we can bump the allocation_ratios for those 3 to 9999.0 in devstack | |
| 19:51:40 | cfriesen | mriedem: we're based on 14.0.6, nothing in 14.0.7 would affect the pci_devices table migration, I don't think. | |
| 19:51:48 | superdan | mriedem: okay I was kinda thinking maybe a warning if those _were_ enabled to say they really should be removed at this point | |
| 19:51:57 | superdan | mriedem: but if this is what you want that's fine | |
| 19:52:27 | cfriesen | mriedem: in our case the "parent_addr" column was fine before the upgrade, but messed up after | |
| 19:52:32 | mriedem | cfriesen: that fix went into 14.0.6 | |
| 19:52:34 | cfriesen | mriedem: but just for one device | |
| 19:52:51 | mriedem | but i suppose maybe you were already past that migration? | |
| 19:52:55 | cfriesen | mriedem: and that fix came from us. :) | |
| 19:53:43 | mriedem | superdan: we'd have to remove them from the default enabled_filters list first | |
| 19:53:50 | mriedem | otherwise all new installs would start out with warnings | |
| 19:54:04 | mriedem | well, as is the case here with CoreFilter | |
| 19:54:05 | mriedem | ... | |
| 19:54:31 | cfriesen | mriedem: we had run that online data migration in Mitaka to convert all the stuff from the previous release, so the DB looked good in Mitaka. So something got messed up either in the offline or online DB migrations going to Newton. | |
| 19:54:50 | cfriesen | could be internal to us, will have to track it down | |
| 19:57:12 | superdan | mriedem: right | |
| 19:57:26 | mriedem | superdan: so this check should probably be seeing if we're a fresh install... | |
| 19:57:35 | mriedem | and not warn if you don't have those filters enabled | |
| 20:25:26 | openstackgerrit | Eric Fried proposed openstack/nova master: Trim the fat from InstanceInfo https://review.openstack.org/471146 | |
| 20:42:47 | cfriesen | are there known issues with 'nova list' pagination in Newton? | |
| 20:45:11 | cfriesen | we seem to have hit a case where "servers_links" is not included even though there are more servers than have been listed so far | |
| 20:48:12 | melwitt | mriedem: replied on the fixed ips quota patch. the NoopQuotaDriver will always 'pass' on the limit check, but the counting will still happen. that's latent, but it's true it gets worse with the series in that more things become CountableResources instead of just a few | |
| 20:55:29 | sdague | mriedem: https://review.openstack.org/#/c/475067 - the mitaka change, is hitting keystone compatiblity issues | |
| 20:56:36 | superdan | um | |
| 20:56:43 | superdan | did gerrit just hang for anyone else? | |
| 20:58:01 | cfriesen | superdan: nope | |
| 20:58:02 | sdague | mriedem / superdan - I'm going to throw this up there as a potential work around - https://review.openstack.org/475073 | |
| 20:58:20 | sdague | if that passes, I'd be fine pushing that through to backport this thing | |
| 20:58:26 | openstackgerrit | Ed Leafe proposed openstack/nova master: Extract custom resource classes from flavors https://review.openstack.org/473627 | |
| 21:27:42 | cheran | @here Can anyone help me getting the filter properties from the spec_obj in mitaka for the ScheduleFilter https://docs.openstack.org/releasenotes/nova/mitaka.html | |
| 21:27:58 | cheran | "Filters internal interface changed using now the RequestSpec NovaObject instead of an old filter_properties dictionary. In case you run out-of-tree filters, you need to modify the host_passes() method to accept a new RequestSpec object and modify the filter internals to use that new object. You can see other in-tree filters for getting the logic or ask for help in #openstack-nova IRC channel." | |
| 21:28:19 | cheran | The request_spec object doesn;t have the filter, so my question is that do I need to add a new filter dict into that? | |
| 21:40:13 | melwitt | cheran: I don't think so. the RequestSpec has a get_scheduler_hint() method you can call for getting the hints related to your filter | |
| 21:43:57 | ihrachys | mriedem, howdy. I see this in newton: http://logs.openstack.org/75/474275/2/check/gate-tempest-dsvm-neutron-dvr-ubuntu-xenial/d4c39c7/logs/screen-n-cpu.txt.gz#_2017-06-15_19_53_17_745 shouldn't it have been fixed by newer libvirt? | |
| 21:44:02 | ihrachys | or don't we have it in newton? | |
| 21:47:52 | ihrachys | oh it seems like devstack does enable it in master only :-x | |