| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-10-03 | |||
| 22:26:29 | efried | There's a real consumer object now | |
| 22:26:41 | mriedem | so the other thing here, | |
| 22:27:01 | efried | the consumer object (a db table row) has a generation we have to update atomically or die. | |
| 22:27:04 | sean-k-mooney | melwitt: i have spawned 350 instance in one request before on newton and it worked fine | |
| 22:27:11 | mriedem | is that because we have a retry decorator on the select_destinations rpc call, if we get MessagingTimeout from the scheduler b/c it takes too long to schedule 1000 instances in a single request, it re-sends the request to the scheduler with the same list of instances | |
| 22:27:36 | melwitt | sean-k-mooney: ack, that's a data point | |
| 22:27:43 | mriedem | so at this point we would have 2 workers trying to create allocations for the same set of instances (consumers) against the same set of providers | |
| 22:27:49 | mriedem | which will stomp all over themselves | |
| 22:27:51 | efried | oh, okay. That'd do it. | |
| 22:28:08 | mriedem | what i'm wondering is if delete_allocation_for_instance should detect the consumer generation conflict and retry | |
| 22:28:24 | mriedem | like we do on claim_resources https://github.com/openstack/nova/blob/master/nova/scheduler/client/report.py#L1744 | |
| 22:31:11 | efried | mriedem: We decided it should not | |
| 22:31:17 | efried | there was a whole long ML thread about it. | |
| 22:31:41 | efried | Because, we said, there really shouldn't be more than one thing acting on instance allocations at once, we said. | |
| 22:32:02 | efried | IMO the bug is that ^ | |
| 22:33:26 | mriedem | yeah i'm writing up the bug now | |
| 22:33:40 | efried | mriedem: Do you need the ML thread? | |
| 22:33:47 | mriedem | no | |
| 22:33:52 | mriedem | https://bugs.launchpad.net/nova/+bug/1795992 | |
| 22:33:53 | openstack | Launchpad bug 1795992 in OpenStack Compute (nova) "retry_select_destinations decorator can make a mess with allocations in placement in a large multi-create request" [Medium,Triaged] | |
| 22:34:24 | efried | Heh. "make a mess". | |
| 22:34:39 | efried | ...fond because they're *memories*. | |
| 22:34:40 | mriedem | total blowout | |
| 22:34:49 | efried | One time in IKEA | |
| 22:34:52 | mriedem | coincidentally, lbragstad is dealing with that right now | |
| 22:35:03 | efried | Oh, did he pop? Good deal. | |
| 22:35:11 | mriedem | black split pea soup coming out of everything | |
| 22:35:23 | mriedem | let me mind meld with him quick | |
| 22:35:40 | melwitt | congrats lbragstad | |
| 22:36:09 | sean-k-mooney | oh before i forget i popped back to say i just found out that kernel 4.16 added a new netdevsim driver that supports among other coolthings sriov. would people be ok with me creating an experimental gate job to test sriov using fedora28? | |
| 22:36:37 | efried | mriedem: Actually, that patch I mentioned before might possibly make the failure happen earlier in the sequence... | |
| 22:36:59 | efried | because surely the allocation is being overwritten | |
| 22:37:13 | efried | though it might be subject to the same window-teeninenss | |
| 23:07:46 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Use long_rpc_timeout in select_destinations RPC call https://review.openstack.org/607735 | |
| 23:07:52 | mriedem | dansmith: ^ | |
| 23:12:50 | mriedem | melwitt: ocata backport here should be ready to go https://review.openstack.org/#/c/605842/ | |
| 23:13:08 | melwitt | ok | |
| 23:24:05 | mriedem | lyarwood: if you want to get these live migration ipv6 changes into the final ocata release before we put it into EM mode you'll need to get the pike and ocata backports fixed up https://review.openstack.org/#/q/I1201db996ea6ceaebd49479b298d74585a78b006 | |
| 23:38:24 | melwitt | TIL unified object string fields are six.text_type i.e. unicode | |
| 23:46:51 | melwitt | do we have any things where we compare strings agnostic to bytes vs unicode in unit tests? | |
| 23:47:21 | melwitt | this test is asserting the api response as a dict | |
| 23:47:47 | melwitt | and if consoleauth served the request, it's bytes strings and if the unified object served the request, it's unicode strings | |
| #openstack-nova - 2018-10-04 | |||
| 00:42:48 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (7) https://review.openstack.org/571992 | |
| 00:43:19 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (8) https://review.openstack.org/571993 | |
| 00:45:10 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in virt/test_block_device.py https://review.openstack.org/566153 | |
| 00:46:19 | openstackgerrit | Sundar Nadathur proposed openstack/nova-specs master: Nova Cyborg interaction specification. https://review.openstack.org/603955 | |
| 00:51:04 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (3) https://review.openstack.org/574104 | |
| 00:51:15 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (4) https://review.openstack.org/574106 | |
| 00:51:28 | openstackgerrit | Takashi NATSUME proposed openstack/nova master: Remove mox in unit/network/test_neutronv2.py (5) https://review.openstack.org/574110 | |
| 03:09:52 | openstackgerrit | Sundar Nadathur proposed openstack/nova-specs master: Nova Cyborg interaction specification. https://review.openstack.org/603955 | |
| 03:11:21 | openstackgerrit | Merged openstack/nova stable/ocata: Fix the help for the disk_weight_multiplier option https://review.openstack.org/607537 | |
| 03:11:27 | openstackgerrit | Merged openstack/nova stable/ocata: Fix host validity check for live-migration https://review.openstack.org/590611 | |
| 03:11:33 | openstackgerrit | Merged openstack/nova stable/queens: stable-only: fix typo in IVS related privsep method https://review.openstack.org/604817 | |
| 03:14:36 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add restrictions on ``updated_at`` when getting migrations https://review.openstack.org/607798 | |
| 03:22:16 | openstackgerrit | Takashi NATSUME proposed openstack/python-novaclient master: Follow up "Fix up userdata argument to rebuild" https://review.openstack.org/607800 | |
| 03:27:01 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add restrictions on ``updated_at`` when getting instance action records https://review.openstack.org/607801 | |
| 03:30:44 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add restrictions on ``updated_at`` when getting migrations https://review.openstack.org/607798 | |
| 04:36:34 | pooja_jadhav | Hi team, anyone knows about the api_sample_tests in detail.. I have some doubts in it. | |
| 05:16:46 | gmann | pooja_jadhav: hi, what is doubt | |
| 05:39:57 | openstackgerrit | Jake Yip proposed openstack/nova master: Add --before to nova-manage db archive_deleted_rows https://review.openstack.org/556751 | |
| 05:56:01 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add restrictions on ``updated_at`` when getting migrations https://review.openstack.org/607798 | |
| 06:12:03 | openstackgerrit | Brin Zhang proposed openstack/nova master: Add restrictions on ``updated_at`` when getting instance action records https://review.openstack.org/607801 | |
| 06:23:34 | openstackgerrit | OpenStack Proposal Bot proposed openstack/nova stable/rocky: Imported Translations from Zanata https://review.openstack.org/604260 | |
| 06:24:54 | pooja_jadhav | gmann: hello, for the simple tenant usage api, if the instance is BFV then local_gb should not consider DISK_GB. that fix I have done. Now I am trying to add functional test for the same. and I am facing issue while creating BFV instance. | |
| 06:26:54 | pooja_jadhav | gmann: In request template i am passing block_device_mappingv2 object. and its trying to cinder from nova and getting Service catelog empty error. Also checked the existing code. they have used stub.out abd CinderFixture and Its not working in my case. | |
| 06:29:51 | pooja_jadhav | *trying to call cinder from nova | |
| 06:30:23 | brinzhang | When create network (https://developer.openstack.org/api-ref/compute/#create-network), What does mean of "injected" and "multi_host"? | |
| 06:31:35 | pooja_jadhav | it should not consider flavor.DISK_GB | |
| 06:38:36 | gmann | pooja_jadhav: api_sample_tests does not call to actual service, they are being mocked using fixture. | |
| 06:39:01 | gmann | pooja_jadhav: so you need to use cinder Fixture and change fixture if needed - https://github.com/openstack/nova/blob/85b36cd2f82ccd740057c1bee08fc722209604ab/nova/tests/fixtures.py#L1380 | |
| 06:42:06 | gmann | pooja_jadhav: same network test and other cinder test does. | |
| 06:43:23 | gmann | pooja_jadhav: you can write the tempest integration test where all actual services and their behaviour is being tested. but for sample tests you can simply stub the expected things from cinder and check nova expected response/behavior | |
| 06:50:13 | pooja_jadhav | gmann: yes, same point i am able to see that, I need to use Cinder Fixture for the same. Tried same thing which is done at [1]https://github.com/openstack/nova/blob/master/nova/tests/functional/api_sample_tests/test_block_device_mapping_boot.py#L23-L27 | |
| 06:51:31 | gmann | pooja_jadhav: yeah, any error you are getting? | |
| 06:51:35 | gmann | you have patch up | |
| 06:53:48 | pooja_jadhav | gmann: I have added same stubs in the https://github.com/openstack/nova/blob/master/nova/tests/functional/api_sample_tests/test_simple_tenant_usage.py and trying to add 4th as BFV instance. | |
| 06:55:34 | pooja_jadhav | gmann: here is the error log http://paste.openstack.org/show/731449/ | |
| 07:00:14 | gmann | pooja_jadhav: this is going on cinder call? did you use Cinder Fixture ? | |
| 07:00:36 | gmann | pooja_jadhav: if you can push your patch in gerrit (WIP though), i can take a look | |
| 07:01:07 | pooja_jadhav | gmann: yes, In setup method, I have added this line : self.useFixture(nova_fixtures.CinderFixture(self)) | |
| 07:01:07 | pooja_jadhav | self.useFixture(nova_fixtures.CinderFixtureNewAttachFlow(self)) | |
| 07:01:58 | gmann | pooja_jadhav: ok, and you are just calling server create with bdmv2 right | |
| 07:02:20 | pooja_jadhav | gmann: yes, right | |
| 07:04:24 | pooja_jadhav | gmann: Also I checked, v2.32 and v2.42, in https://github.com/openstack/nova/blob/master/nova/tests/functional/api_sample_tests/test_servers.py#L189-L196 they are created BFV instances. they are working perfectly fine. Same things trying to use. but not getting success | |
| 07:04:56 | bauzas | good morning nova | |
| 07:06:31 | pooja_jadhav | gmann: How to use this Cinder Fixture?? Am I using correctly in the setup method of the class where I need it ?? | |
| 07:07:10 | gibi | mriedem, efried: I've read back on yesterday's concurrent update. I think one of the goals of the consumer gen patches to make the concurrent update visible and not overwrite allocations so I feel what mriedem saw was a result of that | |
| 07:09:26 | gibi | mriedem, efried: also I agree with efried that if the last patch that updates claim_resources is merged then that parallel allocation request case will stop eariler in the claim_resources when the second claim_resources call for the same consumer fails on the assumption that the consumer does not exists | |
| 07:11:44 | gmann | pooja_jadhav: i think so. but somehow it is not considering it. | |
| 07:12:57 | gmann | pooja_jadhav can you paste the test_simple_tenant_usage.py or if you can push patch up in gerrit then it is easy to debug | |
| 07:22:30 | pooja_jadhav | gmann: http://paste.openstack.org/show/731450/ | |
| 07:29:17 | gmann | pooja_jadhav: you do not need to use both, use CinderFixture only | |
| 07:31:06 | pooja_jadhav | gmann: I have trued by using used CinderFixture only. But still no success L( | |
| 07:31:08 | pooja_jadhav | :( | |
| 07:31:15 | pooja_jadhav | tried* | |
| 07:38:00 | gmann | pooja_jadhav: and the sample file path you changed ? | |
| 07:39:38 | pooja_jadhav | gmann: I am adding sample files inside https://github.com/openstack/nova/tree/85b36cd2f82ccd740057c1bee08fc722209604ab/nova/tests/functional/api_sample_tests/api_samples/os-simple-tenant-usage/v2.40 | |
| 07:40:26 | pooja_jadhav | and passing use_common_server_api_samples=False to _post_server() method so that, It will get the samples from api_samples/os-simple-tenant-usage/v2.40 only. | |
| 07:43:33 | gmann | pooja_jadhav: that's all you are doing perfectly | |
| 07:44:51 | pooja_jadhav | gmann: yes, I will try my best for this. If anything u think I am missing something then please let me know. :) | |