| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-24 | |||
| 16:55:07 | mriedem | savvas: don't worry about it, you're doing great - i obviously don't know how anything works here either :) | |
| 16:55:46 | mriedem | what i'd like to do is know a simple recreate so i can write a tempest test for this | |
| 16:56:04 | kashyap | Does anyone know for sure, about whether the double quotes are *mandatory* when you specify a key-value in a Nova config attribute as following: | |
| 16:56:07 | kashyap | [libvirt] | |
| 16:56:10 | kashyap | hw_machine_type="x86_64=pc-i440fx-rhel7.2.0" | |
| 16:56:12 | savvas | I run 3 cinder volume servers, each with their own AZ. The controller has no storage/default AZ defined | |
| 16:56:20 | savvas | mriedem: as in steps to reproduce this error? | |
| 16:57:10 | mriedem | savvas: i think just even knowing if you can create a volume in cinder w/o specifying AZ if there is a non-default AZ defined, just one | |
| 16:57:29 | jmlowe | mriedem: yeah, I'm going to trip over this later | |
| 16:57:29 | savvas | no, that doesn't work. At this stage I HAVE to specify an AZ | |
| 16:58:04 | mriedem | savvas: huh, ok, figured cinder would allow it if az isn't specified | |
| 16:58:12 | mriedem | smcginnis: jgriffith: ^ is that by design? | |
| 16:58:21 | mriedem | hell it might even be how nova does it but i'm not sure | |
| 16:58:39 | smcginnis | mriedem: Didn't follow everything, but are you saying if AZ is None, Cinder fails the call? | |
| 16:59:44 | mriedem | smcginnis: is there is a non-default AZ defined in cinder | |
| 17:00:03 | mriedem | so if i create az=foo in cinder, | |
| 17:00:10 | mriedem | do i have to create all volumes explicitly in that az now? | |
| 17:00:34 | smcginnis | mriedem: I didn't think so. | |
| 17:00:39 | smcginnis | Checking... | |
| 17:01:39 | smcginnis | So if storage_availability_zone is set to something other than the default, you may need to also update default_availability_zone. Still looking though. | |
| 17:02:19 | savvas | smcginnis: I have both values set on each node running cinder-volume service | |
| 17:02:32 | mriedem | dansmith: are we supposed to be in meeting-3 for cells? | |
| 17:02:39 | mriedem | and the interop guys are going long? | |
| 17:02:43 | smcginnis | savvas: So both are set to unique values, but if None is passed in for AZ it fails? | |
| 17:03:40 | savvas | smcginnis: I have both set to the zone available to the node. I run nova compute and cinder volume on the same host for this purpose. So hv01 would have default/storage_availability_zone set to hv01-zone, hv02 to hv02-zone and so on | |
| 17:04:07 | mriedem | i think that's wrong | |
| 17:04:15 | mriedem | i think default_availability_zone is supposed to be global | |
| 17:04:21 | savvas | as these hypervisors are supposed to be dependent of their local storage, they shouldn't be looking to place volumes in any other zones | |
| 17:04:44 | melwitt | mriedem: yeah on my calendar we should have a cells meeting right now | |
| 17:05:24 | mriedem | that's what storage_availability_zone is for, | |
| 17:05:30 | mriedem | and it overrides default_availability_zone | |
| 17:05:34 | mriedem | is my understanding | |
| 17:05:43 | mriedem | 'the default for new volumes.' | |
| 17:05:43 | mriedem | 'the storage_availability_zone option value is used as ' | |
| 17:05:43 | mriedem | 'Default availability zone for new volumes. If not set, ' | |
| 17:06:42 | smcginnis | savvas: So still looking for details, if None is passed in for AZ it fails? | |
| 17:07:02 | savvas | mriedem: ye but let's say I put default_availability_zone to "nova", wouldn't that result in volumes floating across the nodes? I am trying to isolate the volumes to the nodes that serve the instances | |
| 17:07:14 | savvas | smcginnis: sorry, yes if None is passed for AZ, it fails | |
| 17:07:44 | mriedem | savvas: i think the point is you don't set a default_availability_zone value | |
| 17:07:58 | mriedem | setting to 'nova' and leaving unset is probably handled differently in the cinder code | |
| 17:08:24 | smcginnis | mriedem: Oh yeah, that's cinder-wide, not per-backend. | |
| 17:08:41 | mriedem | right, that's what i said above, it's global | |
| 17:08:45 | savvas | mmm | |
| 17:08:50 | mriedem | storage_availability_zone is per node | |
| 17:08:54 | mriedem | default_availability_zone is global | |
| 17:08:55 | savvas | so this value should not be set on the cinder-volumes | |
| 17:09:02 | mriedem | backend_availability_zone is per backed per nod | |
| 17:09:03 | mriedem | *node | |
| 17:09:10 | mriedem | savvas: i don't think so no | |
| 17:09:17 | mriedem | but, i'm not entirely sure, hence roping in smcginnis | |
| 17:09:20 | djohnsto | mriedem: Any thoughts on how to resolve this bug - seems to be disagreement with the fix 193047 | |
| 17:09:47 | mriedem | djohnsto: which bug? | |
| 17:10:02 | mriedem | oh https://review.openstack.org/#/c/193047/ | |
| 17:10:22 | djohnsto | Yep. | |
| 17:10:49 | smcginnis | savvas: I can try to test this out later, but try removing those conflicting default_availability_zone values. | |
| 17:10:58 | mriedem | djohnsto: in a meeting now, | |
| 17:11:04 | mriedem | that probably needs to be in the mailing list | |
| 17:11:08 | mriedem | as it's a change in behavior | |
| 17:11:10 | djohnsto | Ok | |
| 17:14:07 | savvas | smcginnis: I removed it but it didn't change anything in terms of the volume create command failing when AZ is not defined | |
| 17:14:43 | smcginnis | savvas: OK, will have to look in to it then. | |
| 17:17:59 | savvas | smcginnis: appreciate it | |
| 17:19:36 | mriedem | savvas: you restarted the services after changing the config right? | |
| 17:23:58 | savvas | mriedem: yes | |
| 17:53:08 | mriedem | anyone remember when the guy in the nova onboarding session asked how he knows if an instance variable is a dict or an object or something else? | |
| 17:53:20 | mriedem | same question, but block_device_mapping | |
| 17:53:40 | mriedem | is a list, is it a single thing? dict or object? if object, which type of object? | |
| 17:53:42 | mriedem | super fun | |
| 17:54:31 | melwitt | I remember | |
| 17:54:46 | artom | mdbooth wrote a thing about that | |
| 17:55:19 | artom | http://lists.openstack.org/pipermail/openstack-dev/2016-June/097529.html | |
| 17:55:39 | melwitt | and yeah. generally I've noticed, things are objects until you get into the virt driver, then a lot of things are dicts | |
| 17:55:46 | artom | The fact that you need a novel to understand wtf is concerning, but at least we now have this novel for posterity | |
| 17:57:07 | mriedem | "as there is also something else typically called 'block_device_mapping' which is not a BlockDeviceMapping object." | |
| 17:57:13 | mriedem | or a BlockDeviceMappingList | |
| 17:57:15 | mriedem | which is the case in places | |
| 17:57:43 | mriedem | sometime the actual variable is bdm passed to a method whose formal parameter name is bdms | |
| 17:59:15 | mriedem | ha, wow | |
| 17:59:26 | artom | No no, I want to help :) | |
| 17:59:29 | mriedem | wait, my wrists or yours? | |
| 18:00:00 | artom | Yes. | |
| 18:00:17 | mriedem | i was specifically looking at _prep_block_device | |
| 18:00:24 | mriedem | which i think takes in a BlockDeviceMappingList | |
| 18:00:48 | openstackgerrit | Lars Kellogg-Stedman proposed openstack/nova master: provide interface-scoped nameserver information https://review.openstack.org/467699 | |
| 18:00:50 | mriedem | and through the magic of driver.get_block_device_info(instance, bdms) converst that list to a list of DriverBlockDevice objects | |
| 18:00:57 | mriedem | which are wrappers around a BlockDeviceMapping objec | |
| 18:00:58 | mriedem | *object | |
| 18:22:44 | tidwellr | I'm probably grasping at straws, but is there a chance anyone here can make sense of this error http://paste.openstack.org/show/610594 seen on stable/mitaka? | |
| 18:23:12 | mriedem | gmann: weren't you talking about something with hide addresses earlier today? ^ | |
| 18:25:02 | mriedem | tidwellr: the instance isn't in the req cache for some reason https://github.com/openstack/nova/blob/stable/mitaka/nova/api/openstack/compute/hide_server_addresses.py#L62 | |
| 18:25:34 | tidwellr | mriedem: that's what it looks like to me, the question is why | |
| 18:25:53 | mriedem | it should be cached here https://github.com/openstack/nova/blob/stable/mitaka/nova/api/openstack/compute/servers.py#L443 | |
| 18:25:55 | mriedem | so not sure | |
| 18:26:14 | mriedem | unless the extension's show() method is processed before the instance is cached? | |
| 18:26:21 | mriedem | sdague: ^ any idea how that craziness works? | |
| 18:26:28 | tidwellr | what's weird is the user that reported this can run operations from the CLI just fine, but has an app that invokes the API directly and that is where we see this | |
| 18:27:17 | tidwellr | I'm wondering if it is related to auth/domains somehow, again I'm grasping at straws here | |
| 18:28:43 | sdague | mriedem: I need you to summarize the question | |
| 18:28:49 | mriedem | sdague: heh, | |
| 18:28:58 | mriedem | how do extensions work in the api :) | |