Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-03-23
15:05:45 mriedem Kevin_Zheng: that's why we create the BDM with device_name=None here http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n3678
15:05:48 Kevin_Zheng do something once we got the right name
15:05:55 mriedem otherwise we do http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n3687
15:06:13 mriedem Kevin_Zheng: are you proposing that we set the device_name in the BDM after unshelve?
15:06:20 madhaviy @mriedem, one question on nova/network/neutronv2/api's _check_external_network_attach() method. This is checking whether user/context can attach a network which is marked as external. so when this code is executed during deploy/spawn, policy checking is been done on compute host. Do we need to validate this even before
15:06:24 Kevin_Zheng yeah I did some learning this afternoon
15:06:30 gibi bauzas: there was a proposal from jaypipes to revisit the server group concept here https://review.openstack.org/#/c/183837/4/specs/liberty/approved/generic-scheduling-policies.rst
15:06:37 mriedem Kevin_Zheng: i think that might still be a lie though,
15:06:52 mriedem because just because the user asked for /dev/vda doesn't mean that's where it will show up on the guest
15:06:58 mriedem even though we stored that in cinder
15:07:03 Kevin_Zheng yeah
15:07:16 mriedem this is really why we need to just have a microversion to take device_name out of the volume attach API :)
15:07:21 mriedem because we can't honor it
15:07:27 mriedem we don't even try with the libvirt driver
15:07:29 mriedem we ignore it
15:07:32 Kevin_Zheng yeah
15:07:45 Kevin_Zheng I will try to put up one
15:07:56 Kevin_Zheng if nobody does
15:08:03 mriedem thanks, this is something we've talked about in previous summits and meetups and just never had someone write a spec
15:08:16 mriedem but it comes up as a problem about every 3 months :)
15:08:23 Kevin_Zheng ：）
15:08:37 Kevin_Zheng but still
15:08:58 Kevin_Zheng we will have none in cinder if not provide one
15:09:01 bauzas gibi: yup, I reviewed it even :)
15:09:17 Kevin_Zheng anyway to sync it？
15:09:23 gibi bauzas: I think that spec contains a lot of nice idea and discussion on this matter
15:11:45 mriedem Kevin_Zheng: let me check the cinder api
15:11:58 mriedem if mountpoint is required then we'd have an issue
15:12:52 Kevin_Zheng mriedem: Im using phone so I can't send you the code，but there is a if condation in
15:13:14 Kevin_Zheng nova's attach function
15:13:59 Kevin_Zheng it checked volume.attach_status =detached
15:14:36 Kevin_Zheng maybe we can also check device name there，if none，send again the
15:14:42 Kevin_Zheng right one
15:15:48 openstackgerrit Maciej Szankin proposed openstack/nova-specs master: Add deprecate-nova-cert spec https://review.openstack.org/448882
15:16:49 openstackgerrit Maciej Szankin proposed openstack/nova-specs master: Add deprecate-nova-cert spec https://review.openstack.org/448882
15:19:02 mriedem Kevin_Zheng: looks like the mountpoint is required in the os-attach API in cinder
15:19:23 mriedem in fact, it looks like i can trigger a 500 error if i don't include it in the request body
15:26:13 mriedem melwitt: dansmith: I think Kevin_Zheng found another fun local delete cell0 issue :)
15:26:44 mriedem not totally sure yet, but we don't call http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n2144 when doing the local delete case where the instance isn't in a cell
15:26:56 mriedem so the instance is in error state but has volumes attached, and we don't cleanup the volumes
15:27:19 mriedem i think this would be (1) create server, (2) shelve it, (3) attach volume (4) unshelve - fails, puts instance in error state (5) delete
15:27:40 mriedem the question is, does the instance got to cell0 on the unshelve failure in step 4?
15:28:24 dansmith shouldn't,
15:28:51 dansmith because you can't move the instance, so it wouldn't reschedule to cell0 if it's already in cell1, which it has to be because it was shelved in the first place
15:29:05 mriedem madhaviy: yes the policy check happens on the compute, which is odd, i've brought this up in the mailing list before, i'd have to dig that up
15:29:20 mriedem dansmith: but on shelve offload the instance no longer has a host right?
15:29:25 mriedem althought,
15:29:28 mriedem it should have an instance mapping
15:29:31 mriedem back to the cell it's in
15:29:32 dansmith mriedem: sure, but it's in a cell
15:29:39 mriedem ok
15:29:41 dansmith you can't move the instance record from one db to the other Imeant
15:29:59 mriedem but a shelved offloaded instance will have an instance mapping,
15:30:01 mriedem so it's in a cell
15:30:04 mriedem even if instance.host is NOne
15:30:10 madhaviy mriedem: sure
15:30:18 dansmith right
15:31:05 mriedem so in that case,
15:31:09 mriedem on delete, we still get here http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n1847
15:31:19 mriedem and the cell could be cell1 or whatever
15:31:39 mriedem and we just destroy the instance here http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n1887
15:32:37 mriedem which bypasses http://git.openstack.org/cgit/openstack/nova/tree/nova/compute/api.py#n2206
15:32:48 mriedem if i'm following this correctly
15:33:09 mriedem basically, for local delete we've duplicated the paths, and are missing some things from the case that we don't have instance.host but it's in a cell
15:34:08 mriedem i think i could write a functional test for this pretty easily,
15:34:20 mriedem and verify if the bdm was destroyed when the instance was deleted or not
16:07:38 mriedem weird, the python-novaclient jobs are timing out since 3/21
16:07:49 mriedem http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Killed%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20timeout%20-s%209%5C%22%20AND%20tags%3A%5C%22console%5C%22%20AND%20project%3A%5C%22openstack%2Fpython-novaclient%5C%22&from=7d
16:10:13 openstackgerrit Maciej Szankin proposed openstack/nova-specs master: Add remove-nova-cert spec https://review.openstack.org/448882
16:10:54 openstackgerrit Maciej Szankin proposed openstack/nova-specs master: Add remove-nova-cert spec https://review.openstack.org/448882
16:11:59 sdague mriedem: well there are some crash dump tests in there that are taking 4 minutes to run
16:12:52 sdague mriedem: http://logs.openstack.org/41/449041/1/check/gate-novaclient-dsvm-functional-identity-v3-only-ubuntu-xenial-nv/191f2b4/console.html#_2017-03-23_11_04_17_455195
16:13:07 sdague something else is going on, the tests passed, then zuul forgot about it for an hour
16:14:06 mriedem yeah
16:14:07 mriedem 2017-03-23 11:04:17.455195 | 2017-03-23 11:04:17.449 | Finished
16:14:15 mriedem 2017-03-23 12:00:48.459991 | /home/jenkins/workspace/gate-novaclient-dsvm-functional-identity-v3-only-ubuntu-xenial-nv/devstack-gate/functions.sh: line 1074: 3542 Killed
16:14:22 mriedem so 56 minutes of nothing
16:14:34 sdague I saw a thing like that before and brought it up in infra, I can't remember now what was getting lost
16:21:40 efried harlowja One thing I'm finding with TaskFlow is it's kinda tricky to unit test around.
16:22:07 efried Especially with nested flows and such.
16:23:22 efried If I mock the whole TaskFlow apparatus, I have to test the individual tasks separately and can't get good verification of the overall flow with dependencies and such.
16:23:36 johnthetubaguy tojuvone: are you still about, I forgot to ping you when I got back
16:24:00 tojuvone hieulq, yes, still here
16:24:39 edmondsw johnthetubaguy are you aware that there is a policy rule (network:attach_external_network) being checked by nova-compute?
16:24:42 efried harlowja But if I don't, I better not try to mock any of the individual tasks - too much internal stuff is expected.
16:25:16 efried Not a show-stopper. And not sure what could be done to mitigate - maybe taskflow lib providing some stock fixtures?
16:25:20 sdague mriedem: are there no individual test timeouts on these?
16:25:35 edmondsw johnthetubaguy just found that doesn't work if I don't put a policy file on a host... ugh... we should only be checking policy in nova-api
16:25:43 johnthetubaguy edmondsw: you reminded me, I forgot about that one again, there are a few strange stragglers like that
16:25:53 tojuvone johnthetubaguy, yes, so was wondering the maintenance_events. Wither own table or in services table
16:26:17 tojuvone johnthetubaguy, services exposed to servers so that works
16:26:27 mriedem sdague: i'm not seeing any
16:26:30 edmondsw johnthetubaguy that's gotta be a defect, right? There's even talk about nova-compute being split out from nova. It shouldn't be relying on nova's policy file
16:26:37 tojuvone johnthetubaguy, but in services need to duplicate on each service on host
16:26:39 mriedem sdague: i'm seeing rabbit have memory limit warnings around the time we lose the logs
16:27:19 mriedem edmondsw: see my reply to madhaviy above
16:27:19 sdague yeh, there is no individual test timeout, the test is still running and hung
16:27:36 edmondsw mriedem, ah, didn't realize we'd already had this discussion...
16:27:37 mriedem sdague: doesn't testr have a 500s timeout per test?
16:27:45 johnthetubaguy tojuvone: given what I described, I think it would get its own table, but not sure if everyone likes that direction, I think we should focus on getting the use cases crisp first

Earlier   Later