| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-12-20 | |||
| 11:12:13 | fanzhang | hi, recently we use rally to do a concurrency test booting total 1000 vms which concurrency=45 and times=25. Only very few vms got NoValidHost error because of ('Connection aborted.', BadStatusLine("''",)), then from the scheduler log, we can see "Got no allocation candidates from the Placement API...".but placement service is up abd good. Any ideas why this happens and how to solve it ? | |
| 11:12:32 | fanzhang | s/abd/and | |
| 11:16:35 | fanzhang | poke Kevin_Zheng :) | |
| 11:16:57 | openstackgerrit | Stephen Finucane proposed openstack/nova stable/queens: Handle unbound vif plug errors on compute restart https://review.openstack.org/626550 | |
| 11:20:56 | openstackgerrit | Stephen Finucane proposed openstack/nova stable/pike: Handle unbound vif plug errors on compute restart https://review.openstack.org/626554 | |
| 11:27:01 | openstackgerrit | Stephen Finucane proposed openstack/nova stable/ocata: Handle unbound vif plug errors on compute restart https://review.openstack.org/626556 | |
| 12:42:08 | openstackgerrit | Merged openstack/nova master: Document using service user tokens for long running operations https://review.openstack.org/626388 | |
| 13:12:20 | gibi | stephenfin: hit me with that doc patch that I can approve | |
| 13:12:59 | stephenfin | gibi: Whoops, meant to include a link https://review.openstack.org/#/c/625878/ | |
| 13:13:12 | gibi | stephenfin: looking | |
| 13:40:45 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Document using service user tokens for long running operations https://review.openstack.org/626578 | |
| 13:42:20 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Fix an inaccurate link in nova doc https://review.openstack.org/626579 | |
| 13:43:51 | mriedem | avolkov: are you going to propose backports for https://review.openstack.org/#/c/624870/ ? | |
| 13:45:09 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Fix a broken-link in nova doc https://review.openstack.org/626580 | |
| 13:45:30 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Fix a broken-link in nova doc https://review.openstack.org/626581 | |
| 13:45:59 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Fix a broken-link in nova doc https://review.openstack.org/626580 | |
| 13:47:58 | mriedem | tonyb: when you're awake, debate happening in a proposed backport https://review.openstack.org/#/c/625880/ | |
| 13:48:38 | avolkov | mriedem: yes I can, for what releases? | |
| 13:49:01 | mriedem | avolkov: it goes all the way back to ocata, but if you can do rocky and queens i can do pike and ocata | |
| 13:49:30 | avolkov | mriedem: sure | |
| 13:49:34 | mriedem | thanks | |
| 13:53:17 | openstackgerrit | Andrey Volkov proposed openstack/nova stable/rocky: Exclude build request marker from server listing https://review.openstack.org/626584 | |
| 13:54:47 | openstackgerrit | Andrey Volkov proposed openstack/nova stable/queens: Exclude build request marker from server listing https://review.openstack.org/626585 | |
| 14:03:40 | openstackgerrit | Matt Riedemann proposed openstack/nova stable/rocky: Exclude build request marker from server listing https://review.openstack.org/626584 | |
| 14:30:55 | openstackgerrit | Merged openstack/nova-specs master: Spec: Support filtering by forbidden aggregate https://review.openstack.org/603352 | |
| 14:55:47 | openstackgerrit | Merged openstack/nova-specs master: Expose virtual device tags in REST API https://review.openstack.org/393930 | |
| 14:56:32 | cezary_zukowski | Hey, how to show what physical resources are associated with a particular availability zone? To list AZs, I use `openstack availability zone list` | |
| 15:00:07 | canori01 | mriedem: I got the behavior I wanted with locking the resize operation to an az by piggybacking a force_hosts [instance['availability_zone']] when allow_resize_to_same_host was set. I'm not sure if it's worthwhile to request the feature or if this is too unique to the way I'm running my deployment. Either way, thanks for all your help :) | |
| 15:01:13 | mriedem | cezary_zukowski: you'd likely need to build some tooling based on a few APIs - get the host aggregate for the az, and then from the host aggregate you get the member hosts, and from that you can use the hypervisor-show CLI (or apis) to get resource details for each host in the aggregate (az) | |
| 15:02:17 | mriedem | canori01: so https://docs.openstack.org/nova/latest/admin/availability-zones.html but for resize? | |
| 15:02:43 | mriedem | oh and instance.az would be restricted to whatever the instance is in even if the user didn't request an az | |
| 15:03:55 | canori01 | mriedem: yep! | |
| 15:04:13 | mriedem | canori01: well, you could see if others want similar functionality by sending an email to the openstack-discuss mailing list, prefix the subject with [nova][ops] to filter it properly, and then just explain the use case and how this solves it - and see if others are interested | |
| 15:04:32 | canori01 | Alrighty, thanks! | |
| 15:04:54 | jmlowe | so, let's say I've shoved in a bunch of retry decorators for things like create event, block device mapping update or create, etc because I've had problems with dbdeadlocks dropping transactions, what's the best way to roll those up into a bug and patch? | |
| 15:06:10 | mriedem | jmlowe: some already have that, like action_event_start | |
| 15:06:18 | jmlowe | not enough | |
| 15:06:34 | mriedem | sure, but my point is you might have some covered already | |
| 15:06:47 | jmlowe | I've added about 6 as I've found them in my logs | |
| 15:06:58 | mriedem | so maybe report a bug for your worst offenders | |
| 15:07:21 | mriedem | i'm not sure i'd put them all into the same patch, but idk | |
| 15:07:27 | mriedem | which release? pike? | |
| 15:07:35 | cezary_zukowski | mriedem: `openstack aggregate list` shows nothing to me. The only sensible combination now I see is `openstack host list` (prints: hostname, service, AZ) and `openstack hypervisor list` (prints: hostname, IP). Do you know anything more useful? :) | |
| 15:08:49 | jmlowe | today was block_device_mapping_update_create and migration_create, I did migration_update for good measure because I figured I'd probably hit that as well, afaik there's really no harm in saying if this fails due to deadlock try it a gain a few times | |
| 15:09:09 | mriedem | jmlowe: the migration ones are covered recently too | |
| 15:09:21 | jmlowe | how recently? queens? | |
| 15:09:22 | mriedem | well i thought they were | |
| 15:09:32 | mriedem | maybe i'm thinking of a bug in the gate where we hit that on migratoins (resize) | |
| 15:09:53 | openstack | Launchpad bug 1783402 in OpenStack Compute (nova) "DBDeadlock during test_resize_server_revert_with_volume_attached when setting migration.status='migrating'" [Medium,Confirmed] | |
| 15:09:53 | mriedem | https://bugs.launchpad.net/nova/+bug/1783402 | |
| 15:10:00 | openstack | Launchpad bug 1642537 in OpenStack Compute (nova) "finish_resize fails with DBDeadlock on migrations table" [Undecided,Confirmed] | |
| 15:10:00 | mriedem | https://bugs.launchpad.net/nova/+bug/1642537 | |
| 15:10:08 | mriedem | jmlowe: so there are your existing bugs for deadlocks in the migrations table... :) | |
| 15:10:10 | jmlowe | I'm starting to form the opinion that evernything except get's should be retried | |
| 15:10:59 | mriedem | jmlowe: so how about we ease into this hot tub by you figuring out if (1) are those bugs duplicates and (2) post patches for them (assuming they overlap with what you've already patched locally | |
| 15:11:12 | mriedem | and then follow up with a bug report for the bdm one | |
| 15:11:39 | mriedem | i have to run, but that's my advice to get started | |
| 15:11:47 | mriedem | cezary_zukowski: i'd have to look but can't right now | |
| 15:14:55 | mriedem_afk | cezary_zukowski: shows hosts within an az https://developer.openstack.org/api-ref/compute/?expanded=get-detailed-availability-zone-information-detail#get-detailed-availability-zone-information | |
| 15:15:41 | mriedem_afk | cezary_zukowski: so try this https://docs.openstack.org/python-novaclient/latest/cli/nova.html#nova-availability-zone-list | |
| 15:16:02 | mriedem_afk | or openstack availability zone list --compute | |
| 15:16:06 | mriedem_afk | https://docs.openstack.org/python-openstackclient/latest/cli/command-objects/availability-zone.html#availability-zone-list | |
| 15:16:09 | cezary_zukowski | mriedem_afk: Many thanks | |
| 15:16:29 | mriedem_afk | you'll have to be admin by default to get host info out of those | |
| 15:16:38 | cezary_zukowski | okay | |
| 15:37:24 | openstackgerrit | Merged openstack/nova master: conf: Convert 'live_migration_inbound_addr' to HostAddressOpt https://review.openstack.org/456573 | |
| 15:37:37 | openstackgerrit | Merged openstack/nova master: doc: Rework 'resize' user doc https://review.openstack.org/625878 | |
| 16:00:46 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Transfer port.resource_request to the scheduler https://review.openstack.org/567268 | |
| 16:00:47 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Fill the RequestGroup mapping during schedule https://review.openstack.org/619528 | |
| 16:00:47 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Calculate RequestGroup resource provider mapping https://review.openstack.org/616239 | |
| 16:00:47 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Extend RequestGroup object for mapping https://review.openstack.org/619527 | |
| 16:00:48 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Recalculate request group - RP mapping during re-schedule https://review.openstack.org/619529 | |
| 16:00:48 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Pass resource provider mapping to neutronv2 api https://review.openstack.org/616240 | |
| 16:00:49 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Test boot with more ports with bandwidth request https://review.openstack.org/573317 | |
| 16:00:49 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Send RP uuid in the port binding https://review.openstack.org/569459 | |
| 16:00:50 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reject networks with QoS policy https://review.openstack.org/570079 | |
| 16:00:50 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Reject interface attach with QoS aware port https://review.openstack.org/570078 | |
| 16:00:51 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Remove port allocation during detach https://review.openstack.org/622421 | |
| 16:00:52 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Add pf_interface_name tag to passthrough_whitelist https://review.openstack.org/625311 | |
| 16:00:52 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Record requester in the InstancePCIRequest https://review.openstack.org/625310 | |
| 16:00:52 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Refactor PortResourceRequestBasedSchedulingTestBase https://review.openstack.org/624080 | |
| 16:00:53 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Ensure that bandwidth and VF are from the same PF https://review.openstack.org/623543 | |
| 16:28:25 | openstackgerrit | Merged openstack/os-vif master: Change openstack-dev to openstack-discuss https://review.openstack.org/621825 | |
| 16:37:55 | cfriesen | sean-k-mooney_: you around? could you take another look at jackding's cpu model spec? | |
| 16:38:15 | sean-k-mooney_ | technically im on vacation but yes | |
| 16:38:26 | sean-k-mooney_ | i can take a look before i sign off | |
| 16:39:00 | sean-k-mooney_ | i have a pep8 issue on my own spec so i need to respin it again before i disapaer | |
| 16:42:44 | adrianc | sean-k-mooney_: Enjoy your vacation :) coming back after new year ? | |
| 16:42:48 | gibi | mriedem: I'm soon leave for the year. I don't know if you see the discussion in the ML thread about the feature flag. There are some complications. I introduced one possible solution to the patch that enables the boot with bandwidth | |
| 16:43:05 | sean-k-mooney_ | adrianc: ya the 3rd | |
| 16:43:31 | gibi | mriedem: anyhow I'm sure we will continue talking about next year. (I'm back on 7th Jan) | |
| 16:44:21 | sean-k-mooney_ | i may be online intermitently and take some time in excange after m2 if it makes sese but effectly im away for the next 2 weeks | |
| 16:48:55 | mriedem | gibi: i haven't gone through the ML thread yet, but i will, | |
| 16:49:13 | mriedem | gibi: in general i'm not supportive of feature flags in the API (smells like config-driven API behavior) | |
| 16:49:26 | mriedem | unless it's policy or something | |
| 16:49:38 | mriedem | gibi: coincidentally, i will be on vacation the week of jan 7 :) | |
| 16:50:53 | gibi | mriedem: if you leave me reasonable suggestion I can try to implement that on the week of Jan you are off | |
| 16:51:30 | mriedem | i'll make it your xmas gift | |
| 16:51:35 | gibi | mriedem: thanks | |
| 16:51:37 | gibi | :) | |