| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-06-25 | |||
| 16:47:31 | efried | mriedem: ack, I like it. | |
| 16:48:08 | stephenfin | Actually, I want to test this more so I'll leave it til the morning | |
| 16:48:46 | stephenfin | efried: Sort of. I'm going to drop the first patch and then address the open TODOs in the remainder. If you want to take a look at any you haven't reviewed yet though, that could be helpful | |
| 16:49:38 | efried | something weird just happened | |
| 16:50:04 | stephenfin | Ohh, that's interesting | |
| 16:50:18 | stephenfin | I Ctrl + C'd git-review during the upload once I realized I'd more work to do | |
| 16:50:40 | efried | bot didn't notify, and when I look at the review it seems... corrupted. | |
| 16:51:23 | efried | the series isn't showing up on the right | |
| 16:51:32 | stephenfin | Feck it, I'll push it again just to fix that | |
| 16:51:46 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Privsepify ipv4 forwarding enablement. https://review.opendev.org/635431 | |
| 16:51:47 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Privsep the ebtables modification code. https://review.opendev.org/635435 | |
| 16:51:47 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove unused FP device creation and deletion methods. https://review.opendev.org/635433 | |
| 16:51:48 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move iptables rule fetching and setting to privsep. https://review.opendev.org/636508 | |
| 16:51:48 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move adding vlans to interfaces to privsep. https://review.opendev.org/635436 | |
| 16:51:49 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move router advertisement daemon restarts to privsep. https://review.opendev.org/639281 | |
| 16:51:49 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move dnsmasq restarts to privsep. https://review.opendev.org/639280 | |
| 16:51:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move setting of device trust to privsep. https://review.opendev.org/639283 | |
| 16:51:50 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move calls to ovs-vsctl to privsep. https://review.opendev.org/639282 | |
| 16:51:51 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Cleanup the _execute shim in nova/network. https://review.opendev.org/639581 | |
| 16:51:51 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Move final bridge commands to privsep. https://review.opendev.org/639580 | |
| 16:51:52 | openstackgerrit | Stephen Finucane proposed openstack/nova master: Cleanup no longer required filters and add a release note. https://review.opendev.org/639826 | |
| 16:51:52 | openstackgerrit | Stephen Finucane proposed openstack/nova master: We no longer need rootwrap. https://review.opendev.org/554438 | |
| 16:53:06 | efried | that seems to have worked. | |
| 16:53:10 | stephenfin | Indeed | |
| 16:53:12 | stephenfin | and with that | |
| 16:53:20 | efried | why did you drop the bottom patch? | |
| 16:53:21 | sean-k-mooney | thats strang it should not have currpted it | |
| 16:53:52 | stephenfin | efried: It seemed like you didn't want it https://review.opendev.org/#/c/655282/ | |
| 16:54:10 | efried | stephenfin: just one piece of it. The rest was good. | |
| 16:54:19 | efried | anyway, can do it elsewhere in the series | |
| 16:54:23 | stephenfin | Ah, then I can come back and stick that back in higher up in the series | |
| 16:54:31 | efried | ++ thanks stephenfin | |
| 17:11:55 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Implement update_provider_tree https://review.opendev.org/667417 | |
| 17:24:24 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Clean up test_virtapi https://review.opendev.org/667419 | |
| 17:44:27 | openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP libvirt: Add rbd_connect_timeout configurable https://review.opendev.org/667421 | |
| 18:05:53 | openstackgerrit | Ghanshyam Mann proposed openstack/nova master: Multiple API cleanup changes https://review.opendev.org/666889 | |
| 18:29:29 | openstackgerrit | Matt Riedemann proposed openstack/nova master: xenapi: implement update_provider_tree https://review.opendev.org/667435 | |
| 18:32:06 | openstackgerrit | Matt Riedemann proposed openstack/nova master: xenapi: implement update_provider_tree https://review.opendev.org/667435 | |
| 18:39:06 | openstackgerrit | Kashyap Chamarthy proposed openstack/nova stable/rocky: libvirt: Rework 'EBUSY' (SIGKILL) error handling code path https://review.opendev.org/667436 | |
| 18:54:13 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Deprecate non-update_provider_tree compat code https://review.opendev.org/667442 | |
| 18:55:40 | mriedem | efried: thoughts on only logging the warning on startup in ^ or every update_available_resource periodic? | |
| 18:55:58 | mriedem | the latter would be really chatty, but maybe that's the level of annoying we want | |
| 19:11:46 | NewBruce | 2019-06-25 20:47:45.302 166805 WARNING nova.virt.libvirt.driver [req-bda9173a-31ed-447f-bf57-3ad85c565141 - - - - -] [instance: 5123fa51-512f-4f13-ae4b-5bbc7dd64e94] Error monitoring migration: Compute host cc-compute04-sto2 could not be found.: ComputeHostNotFound: Compute host cc-compute04-sto2 could not be found. | |
| 19:11:46 | NewBruce | 2019-06-25 20:47:45.167 166805 INFO nova.compute.manager [req-bda9173a-31ed-447f-bf57-3ad85c565141 - - - - -] [instance: 5123fa51-512f-4f13-ae4b-5bbc7dd64e94] Migrating instance to cc-compute25-sto2 finished successfully. | |
| 19:11:46 | NewBruce | evening all! mriedem have you seen this issue before mate? | |
| 19:12:04 | NewBruce | got me stumped…. and why always me and migration issues? :) | |
| 19:12:31 | NewBruce | the instance is migrated, but is left in status ERROR | |
| 19:21:34 | NewBruce | ValueError: year is out of range | |
| 19:21:34 | NewBruce | raise ValueError(six.text_type(e)) | |
| 19:21:34 | NewBruce | File "/openstack/venvs/nova-18.0.0/lib/python2.7/site-packages/oslo_utils/timeutils.py", line 68, in parse_isotime | |
| 19:21:34 | NewBruce | Not sure if its related, but the source node is throwing this error… a lot | |
| 19:36:51 | Nick_A | why is nova (?) metadata sending all /24 routes in a region to each instance? http://paste.openstack.org/show/y0lE42EA59yhnu7G1KnY/ | |
| 19:39:01 | mriedem | bauzas: dansmith: i take it from the comments here https://github.com/openstack/nova/blob/master/nova/objects/compute_node.py#L150 that one shouldn't rely on ComputeNode.service_id is that correct? | |
| 19:39:13 | mriedem | the Service object has a compute_node field, and ComputeNode has a nullable service_id | |
| 19:39:23 | mriedem | but it looks like neither should really be used... | |
| 19:39:35 | dansmith | um, I dunno | |
| 19:40:35 | mriedem | ala https://github.com/openstack/nova/commit/551be2c52a29cb2755de4825a3fcb2c8f7d7b3f1 and https://github.com/openstack/nova/commit/0b177f18264ca842e16c2c989a081e773b3d86b7 | |
| 19:40:44 | mriedem | ok, lots of really old compat notes from bauzas in all of that code | |
| 19:41:02 | mriedem | going back to kilo/liberty era | |
| 19:41:09 | mriedem | wonder if we'll ever clean that stuff up | |
| 19:41:11 | dansmith | yeah, | |
| 19:41:27 | dansmith | I'd think that that comment is probably old and that a migration fixed the ambiguity long ago, but.. I haven't looked | |
| 19:43:51 | mriedem | https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/models.py#L123 | |
| 19:43:58 | mriedem | i don't think there was ever a migration | |
| 19:44:02 | mriedem | just lots of todos from ~liberty | |
| 19:45:57 | dansmith | hrm | |
| 19:45:59 | mriedem | ok i'd think the migration would be pretty simple, | |
| 19:46:08 | mriedem | iterate cells looking for compute_nodes where host is null | |
| 19:46:18 | mriedem | if so, find their service record and update the compute_nodes.host value | |
| 19:46:55 | mriedem | or a migration run per-cell (we haven't really be consistent there) | |
| 19:46:59 | dansmith | or just have computes update compute nodes on RT update | |
| 19:47:10 | dansmith | no reason to make it a manual-only thing | |
| 19:48:00 | mriedem | yeah i don't think it's even possible to get through RT with a cn that doesn't have a host value set | |
| 19:48:14 | mriedem | we lookup the cn by host and node and if it doesn't exist we create it and always have those values set now | |
| 19:48:34 | mriedem | so...maybe this is as good as a simple nova-status upgrade check to sniff out anything lingering and in U drop all of this compat code | |
| 19:48:43 | dansmith | sure | |
| 19:48:48 | mriedem | not sure we'd want to do the db alter table statements to make the column non-nullable... | |
| 19:48:54 | mriedem | but that's less of an issue | |
| 20:03:06 | efried | mriedem: upt also gets called on spawn & delete, doesn't it? | |
| 20:03:23 | efried | via update_available_resource | |
| 20:05:36 | mriedem | efried: i don't think so, you're thinking of instance_claim | |
| 20:05:58 | efried | okay | |
| 20:06:19 | mriedem | and rt.update_usage | |
| 20:06:57 | mriedem | ^ is the one that has the vm_state check for DELETED/SHELVED_OFFLOADED on the instance and if it's one of those it removes the usage | |
| 20:07:34 | mriedem | looks like _post_live_migration also hits update_available_resource for kicks | |
| 20:08:04 | mriedem | presumably b/c we don't do rt claims for live migration | |
| 20:12:25 | efried | okay, so chattier than just once every 10 minutes. | |
| 20:14:02 | sean-k-mooney | mriedem: well we do in some cases | |
| 20:14:51 | sean-k-mooney | mriedem: we claim pci devices for the sriov case and we will claim cpus and hugepages once artoms numa migration code lands | |
| 20:15:10 | mriedem | update_available_resource is once per minute i think | |
| 20:15:31 | mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.update_resources_interval | |
| 20:15:55 | mriedem | https://github.com/openstack/oslo.service/blob/master/oslo_service/periodic_task.py#L31 | |
| 20:16:04 | sean-k-mooney | its configurable in the config but i thought it was 300 seconds | |
| 20:16:38 | mriedem | i think you're both high on goofballs | |
| 20:17:00 | mriedem | efried: so every minute by default | |
| 20:17:21 | sean-k-mooney | on no its 60s you are right https://docs.openstack.org/nova/latest/configuration/config.html#scheduler.periodic_task_interval | |
| 20:18:26 | sean-k-mooney | oh wait is that only set for the schduler ? i though we had a compute node version too? | |
| 20:24:42 | mriedem | what are you asking | |
| 20:24:59 | mriedem | https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.update_resources_interval is used by the ComputeManager.update_available_resource periodic task | |
| 20:40:13 | openstackgerrit | Eric Fried proposed openstack/nova-specs master: Re-propose the spec to allow specifying a list of CPU models https://review.opendev.org/642030 | |