Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-08-02
19:03:58 mriedem numa_topology, pci_requests?
19:05:32 eandersson numa, nothing sepcial with pci
19:06:02 eandersson I don't think we have anything special in image metadata, will need to double check
19:06:08 mriedem hmm, the numa_topology object gets serialized in that thing as well, and that's likely not something we have in-tree functional tests for (instance + numa + resize reschedule)
19:09:00 mriedem looking at the rocky code, on the first host attempt, conductor will serialize the request spec and pass it to compute, but compute won't pass it back to conductor on reschedule, so conductor has to build a new one https://github.com/openstack/nova/blob/stable/rocky/nova/conductor/manager.py#L304
19:09:20 mriedem which will again get serialized and deserialized to dict form when calling prep_resize for the alternate host
19:09:58 mriedem but the point being in rocky the request used for the first host isn't necessarily the same as the one for the alternate which could mean....something
19:25:17 donnyd is max_concurernt_builds per hypervisor?
19:28:17 donnyd for instance if i want to have only 2 instances per hypervisor schedule at any one point in time I would set that in nova-compute under max_concurernt_builds correct?
19:40:13 eandersson donnyd, yes
19:40:26 eandersson https://github.com/openstack/nova/blob/master/nova/conf/compute.py#L610
19:43:41 donnyd thats what i thought, just confirming
19:43:45 donnyd thanks eandersson
19:48:23 mriedem cdent: looking at osprofiler data payed off https://bugs.launchpad.net/nova/+bug/1838817
19:48:23 openstack Launchpad bug 1838817 in OpenStack Compute (nova) "neutron: refreshing vif model for a server gets the same network multiple times" [Medium,Triaged] - Assigned to Matt Riedemann (mriedem)
20:04:12 eandersson mriedem, we had to set heal_instance_info_cache high because it was killing our control plane
20:04:15 eandersson Is that why? :D
20:04:45 mriedem that's not the only thing,
20:05:07 mriedem but yeah it's expensive b/c building the instance nw info model means calling neutron separately for ports, networks, ips and subnets
20:05:17 mriedem since there isn't a batch api to get all of that information
20:05:34 eandersson It was getting real heavy on large sites with 1k nodes
20:06:10 mriedem yeah b/c each node is hitting that refresh every minute
20:06:20 eandersson We also ended up adding a variance
20:06:39 mriedem a doc
20:07:01 eandersson Yea - let me know we have a lot of custom settings
20:07:12 mriedem as does cern and others
20:07:21 mriedem this kind of stuff comes up and we don't have a place to dump it really in the docs
20:07:26 eandersson Yep
20:07:29 mriedem similar with the long_rpc_timeout option
20:07:38 eandersson Yea - we have it set to 300 :D
20:07:43 eandersson 1k for dhcp agent lol
20:08:22 eandersson (due to this bug https://github.com/openstack/neutron/commit/f9f6ae9c98a9ff9d3c0807b9fd51fb2ffe25fd5d)
20:09:41 mriedem which thing do you have set to 300?
20:09:45 mriedem heal instance info cache interval?
20:10:01 eandersson 240 + variance on heal instance infop
20:10:17 eandersson rcp_timeout is set to 300, but that could have been a mitaka thing
20:11:23 mriedem rpc_response_timeout is the old one that defaults to 60
20:11:33 mriedem rpc_long_timeout is new as of i think rocky and is only used for certain long rpc calls
20:11:37 mriedem like during live migratoin
20:11:43 eandersson ah I see - didn't know about that one at all
20:11:50 mriedem https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.long_rpc_timeout
20:12:02 eandersson Probably another thing we missed when jumping to so many versions.
20:12:16 eandersson ah yea that fixes our problem
20:12:22 eandersson We set it to 300 due to scheduling
20:12:43 mriedem yup, that's using long_rpc_timeout now (select_destinations)
20:13:07 mriedem long_rpc_timeout heartbeats the service every $rpc_response_timeout but the overall call timeout is now long_rpc_timeout
20:13:15 mriedem i opened https://bugs.launchpad.net/nova/+bug/1838819 if you want to dump some stuff in there
20:13:15 openstack Launchpad bug 1838819 in OpenStack Compute (nova) "Docs needed for tunables at large scale" [Undecided,New]
20:13:48 eandersson It would probably be worth to get together during the next summit as well and talk about this (or maybe just a irc meeting)
20:14:32 eandersson I tend to reach out to CERN a lot since they are running similar scale
20:15:21 mriedem even dumping on the ML - lots of people won't be in shanghai
20:15:31 eandersson Yea - including myself
20:15:32 mriedem i'll throw that in hte ML quick
20:15:47 eandersson Not sure why but I don't like emails
20:15:59 eandersson Maybe I'll just join with my private email
20:16:00 mriedem fire and forget...
20:16:15 mriedem i'm ramping down to leave town for the weekend so i'm trying to fire things off
20:17:13 eandersson Same
20:17:26 eandersson It's why I am deep diving into some of the issues.
20:32:31 openstackgerrit Matt Riedemann proposed openstack/nova master: WIP: neutron: avoid redundant show_network calls in _get_physnet_tunneled_info https://review.opendev.org/674422
23:21:46 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Always set a global_request_id in RequestContext https://review.opendev.org/674138
23:22:10 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Always set a global_request_id in RequestContext https://review.opendev.org/674138
23:27:34 openstackgerrit Merged openstack/os-resource-classes master: Sync Sphinx requirement https://review.opendev.org/674164
#openstack-nova - 2019-08-03
01:07:12 openstackgerrit Dustin Cowles proposed openstack/nova master: Provider config file schema and loader https://review.opendev.org/673341
08:49:18 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Filter migrations by user_id/project_id https://review.opendev.org/674243
08:50:43 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Add user_id and project_id colume to Migration https://review.opendev.org/673990
09:00:56 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Add user_id and project_id colume to Migration https://review.opendev.org/673990
09:59:28 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Filter migrations by user_id/project_id https://review.opendev.org/674243
10:12:01 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Filter migrations by user_id/project_id https://review.opendev.org/674243
10:17:08 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Filter migrations by user_id/project_id https://review.opendev.org/674243
12:46:28 openstackgerrit Matt Riedemann proposed openstack/nova master: Make nova-multi-cell job voting and gating https://review.opendev.org/674453
14:43:07 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Add user_id and project_id colume to Migration https://review.opendev.org/673990
#openstack-nova - 2019-08-04
04:06:10 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Add user_id and project_id colume to Migration https://review.opendev.org/673990
05:56:44 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Filter migrations by user_id/project_id https://review.opendev.org/674243
06:09:20 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Filter migrations by user_id/project_id https://review.opendev.org/674243
14:05:25 zhouyao @sean-k-mooney May i ask if you could check this patch? https://review.opendev.org/#/c/621646
20:22:56 openstackgerrit Stephen Finucane proposed openstack/nova master: trivial: Rewrap definitions of 'NUMACell' https://review.opendev.org/674395
20:22:57 openstackgerrit Stephen Finucane proposed openstack/nova master: libvirt: Remove unnecessary argument https://review.opendev.org/674396
20:22:57 openstackgerrit Stephen Finucane proposed openstack/nova master: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800
20:22:58 openstackgerrit Stephen Finucane proposed openstack/nova master: Add support translating CPU policy extra specs, image meta https://review.opendev.org/671801
20:22:58 openstackgerrit Stephen Finucane proposed openstack/nova master: objects: Rename 'fields' import to 'obj_fields' https://review.opendev.org/674103
20:22:59 openstackgerrit Stephen Finucane proposed openstack/nova master: objects: Add proxy fields to NUMATopology object https://review.opendev.org/674104
23:57:32 openstackgerrit Takashi NATSUME proposed openstack/nova master: Filter UnsupportedServiceVersion warning https://review.opendev.org/674515
23:59:33 openstackgerrit Takashi NATSUME proposed openstack/nova master: Add a live migration regression test https://review.opendev.org/641200
23:59:52 openstackgerrit Takashi NATSUME proposed openstack/nova master: Retrun 400 if invalid query parameters are specified https://review.opendev.org/670440
#openstack-nova - 2019-08-05
00:59:25 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Filter migrations by user_id/project_id https://review.opendev.org/674243
01:01:32 openstackgerrit Brin Zhang proposed openstack/nova master: WIP: Add user_id and project_id colume to Migration https://review.opendev.org/673990
03:44:20 openstackgerrit ya.wang proposed openstack/nova master: Support report driver migrate feature https://review.opendev.org/674522
03:44:21 openstackgerrit ya.wang proposed openstack/nova master: Add property check when create/rebuild/resize. https://review.opendev.org/674523
03:44:21 openstackgerrit ya.wang proposed openstack/nova master: Scheduelr translate properties to traits https://review.opendev.org/674524
03:44:22 openstackgerrit ya.wang proposed openstack/nova master: Change method: _live_migration_flags https://review.opendev.org/674525
03:44:22 openstackgerrit ya.wang proposed openstack/nova master: Add releasenote and update docs. https://review.opendev.org/674526
06:11:15 alex_xu bauzas: dansmith looking for some feedback on https://review.opendev.org/#/q/status:open+project:openstack/nova+branch:master+topic:claim_for_instance, both mdev and vpmem are implement
06:45:52 openstackgerrit Yongli He proposed openstack/nova master: Add server sub-resource topology API https://review.opendev.org/621476
07:14:48 alex_xu dansmith: efried, one more corner case found. For same host resize, we only have one domain xml in the libvirt before confirming the resize. So if the host is down, and then the host back again, we will lose the source allocation.
07:16:09 alex_xu dannins: efried, one method is we write the device allocations(vgpu, vpmem..) into the filesystem under the dir "instance_base_path + '_resize'"...
07:25:15 alex_xu dansmith: efried or we just auto confrim that same host resize after the host back, and cleanup other free vpmem.
08:48:15 openstackgerrit ya.wang proposed openstack/nova master: vCPU mdoels selection https://review.opendev.org/670298
08:48:16 openstackgerrit ya.wang proposed openstack/nova master: Add compatibility checks for CPU mode and CPU models and extra flags https://review.opendev.org/670299
08:48:16 openstackgerrit ya.wang proposed openstack/nova master: Support report multi CPU model traits https://review.opendev.org/670300

Earlier   Later