| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-05-24 | |||
| 17:02:32 | mriedem | dansmith: are we supposed to be in meeting-3 for cells? | |
| 17:02:39 | mriedem | and the interop guys are going long? | |
| 17:02:43 | smcginnis | savvas: So both are set to unique values, but if None is passed in for AZ it fails? | |
| 17:03:40 | savvas | smcginnis: I have both set to the zone available to the node. I run nova compute and cinder volume on the same host for this purpose. So hv01 would have default/storage_availability_zone set to hv01-zone, hv02 to hv02-zone and so on | |
| 17:04:07 | mriedem | i think that's wrong | |
| 17:04:15 | mriedem | i think default_availability_zone is supposed to be global | |
| 17:04:21 | savvas | as these hypervisors are supposed to be dependent of their local storage, they shouldn't be looking to place volumes in any other zones | |
| 17:04:44 | melwitt | mriedem: yeah on my calendar we should have a cells meeting right now | |
| 17:05:24 | mriedem | that's what storage_availability_zone is for, | |
| 17:05:30 | mriedem | and it overrides default_availability_zone | |
| 17:05:34 | mriedem | is my understanding | |
| 17:05:43 | mriedem | 'Default availability zone for new volumes. If not set, ' | |
| 17:05:43 | mriedem | 'the storage_availability_zone option value is used as ' | |
| 17:05:43 | mriedem | 'the default for new volumes.' | |
| 17:06:42 | smcginnis | savvas: So still looking for details, if None is passed in for AZ it fails? | |
| 17:07:02 | savvas | mriedem: ye but let's say I put default_availability_zone to "nova", wouldn't that result in volumes floating across the nodes? I am trying to isolate the volumes to the nodes that serve the instances | |
| 17:07:14 | savvas | smcginnis: sorry, yes if None is passed for AZ, it fails | |
| 17:07:44 | mriedem | savvas: i think the point is you don't set a default_availability_zone value | |
| 17:07:58 | mriedem | setting to 'nova' and leaving unset is probably handled differently in the cinder code | |
| 17:08:24 | smcginnis | mriedem: Oh yeah, that's cinder-wide, not per-backend. | |
| 17:08:41 | mriedem | right, that's what i said above, it's global | |
| 17:08:45 | savvas | mmm | |
| 17:08:50 | mriedem | storage_availability_zone is per node | |
| 17:08:54 | mriedem | default_availability_zone is global | |
| 17:08:55 | savvas | so this value should not be set on the cinder-volumes | |
| 17:09:02 | mriedem | backend_availability_zone is per backed per nod | |
| 17:09:03 | mriedem | *node | |
| 17:09:10 | mriedem | savvas: i don't think so no | |
| 17:09:17 | mriedem | but, i'm not entirely sure, hence roping in smcginnis | |
| 17:09:20 | djohnsto | mriedem: Any thoughts on how to resolve this bug - seems to be disagreement with the fix 193047 | |
| 17:09:47 | mriedem | djohnsto: which bug? | |
| 17:10:02 | mriedem | oh https://review.openstack.org/#/c/193047/ | |
| 17:10:22 | djohnsto | Yep. | |
| 17:10:49 | smcginnis | savvas: I can try to test this out later, but try removing those conflicting default_availability_zone values. | |
| 17:10:58 | mriedem | djohnsto: in a meeting now, | |
| 17:11:04 | mriedem | that probably needs to be in the mailing list | |
| 17:11:08 | mriedem | as it's a change in behavior | |
| 17:11:10 | djohnsto | Ok | |
| 17:14:07 | savvas | smcginnis: I removed it but it didn't change anything in terms of the volume create command failing when AZ is not defined | |
| 17:14:43 | smcginnis | savvas: OK, will have to look in to it then. | |
| 17:17:59 | savvas | smcginnis: appreciate it | |
| 17:19:36 | mriedem | savvas: you restarted the services after changing the config right? | |
| 17:23:58 | savvas | mriedem: yes | |
| 17:53:08 | mriedem | anyone remember when the guy in the nova onboarding session asked how he knows if an instance variable is a dict or an object or something else? | |
| 17:53:20 | mriedem | same question, but block_device_mapping | |
| 17:53:40 | mriedem | is a list, is it a single thing? dict or object? if object, which type of object? | |
| 17:53:42 | mriedem | super fun | |
| 17:54:31 | melwitt | I remember | |
| 17:54:46 | artom | mdbooth wrote a thing about that | |
| 17:55:19 | artom | http://lists.openstack.org/pipermail/openstack-dev/2016-June/097529.html | |
| 17:55:39 | melwitt | and yeah. generally I've noticed, things are objects until you get into the virt driver, then a lot of things are dicts | |
| 17:55:46 | artom | The fact that you need a novel to understand wtf is concerning, but at least we now have this novel for posterity | |
| 17:57:07 | mriedem | "as there is also something else typically called 'block_device_mapping' which is not a BlockDeviceMapping object." | |
| 17:57:13 | mriedem | or a BlockDeviceMappingList | |
| 17:57:15 | mriedem | which is the case in places | |
| 17:57:43 | mriedem | sometime the actual variable is bdm passed to a method whose formal parameter name is bdms | |
| 17:59:15 | mriedem | ha, wow | |
| 17:59:26 | artom | No no, I want to help :) | |
| 17:59:29 | mriedem | wait, my wrists or yours? | |
| 18:00:00 | artom | Yes. | |
| 18:00:17 | mriedem | i was specifically looking at _prep_block_device | |
| 18:00:24 | mriedem | which i think takes in a BlockDeviceMappingList | |
| 18:00:48 | openstackgerrit | Lars Kellogg-Stedman proposed openstack/nova master: provide interface-scoped nameserver information https://review.openstack.org/467699 | |
| 18:00:50 | mriedem | and through the magic of driver.get_block_device_info(instance, bdms) converst that list to a list of DriverBlockDevice objects | |
| 18:00:57 | mriedem | which are wrappers around a BlockDeviceMapping objec | |
| 18:00:58 | mriedem | *object | |
| 18:22:44 | tidwellr | I'm probably grasping at straws, but is there a chance anyone here can make sense of this error http://paste.openstack.org/show/610594 seen on stable/mitaka? | |
| 18:23:12 | mriedem | gmann: weren't you talking about something with hide addresses earlier today? ^ | |
| 18:25:02 | mriedem | tidwellr: the instance isn't in the req cache for some reason https://github.com/openstack/nova/blob/stable/mitaka/nova/api/openstack/compute/hide_server_addresses.py#L62 | |
| 18:25:34 | tidwellr | mriedem: that's what it looks like to me, the question is why | |
| 18:25:53 | mriedem | it should be cached here https://github.com/openstack/nova/blob/stable/mitaka/nova/api/openstack/compute/servers.py#L443 | |
| 18:25:55 | mriedem | so not sure | |
| 18:26:14 | mriedem | unless the extension's show() method is processed before the instance is cached? | |
| 18:26:21 | mriedem | sdague: ^ any idea how that craziness works? | |
| 18:26:28 | tidwellr | what's weird is the user that reported this can run operations from the CLI just fine, but has an app that invokes the API directly and that is where we see this | |
| 18:27:17 | tidwellr | I'm wondering if it is related to auth/domains somehow, again I'm grasping at straws here | |
| 18:28:43 | sdague | mriedem: I need you to summarize the question | |
| 18:28:49 | mriedem | sdague: heh, | |
| 18:28:58 | mriedem | how do extensions work in the api :) | |
| 18:29:07 | artom | mriedem, what I'm seeing is that the instance is obtained here: https://github.com/openstack/nova/blob/stable/mitaka/nova/api/openstack/compute/hide_server_addresses.py#L62 | |
| 18:29:24 | mriedem | artom: it has to be None | |
| 18:29:26 | artom | Is that from cache? Or is it missing from the bd | |
| 18:29:32 | mriedem | it's not in the cache | |
| 18:29:39 | mriedem | AttributeError: 'NoneType' object has no attribute 'get' | |
| 18:30:01 | mriedem | sdague: do the servers extensions for show() happen after the actual servers controller is called? | |
| 18:30:22 | sdague | yes, iirc. This is mitaka? | |
| 18:30:23 | mriedem | sdague: i.e. we do GET /servers/{server_id} and store the instance in the request cache here https://github.com/openstack/nova/blob/stable/mitaka/nova/api/openstack/compute/servers.py#L443 | |
| 18:30:26 | mriedem | yeah | |
| 18:30:33 | sdague | that's my recollection | |
| 18:30:43 | sdague | tidwellr: what was going on before the stack trace? | |
| 18:31:23 | mriedem | i see other server extensions like for security groups handle this differently | |
| 18:32:54 | mriedem | every other extension does something like https://github.com/openstack/nova/blob/stable/mitaka/nova/api/openstack/compute/servers.py#L443 | |
| 18:32:56 | mriedem | oops | |
| 18:33:01 | mriedem | req.get_db_instance(server['id']) | |
| 18:33:10 | mriedem | they don't use the id parameter in the 'show' method | |
| 18:33:10 | tidwellr | sdague: I've got a user who is making calls to v2.1/<project ID>/servers/<server ID> | |
| 18:33:25 | tidwellr | in this case, it's a GET | |
| 18:33:27 | sdague | tidwellr: what kind of calls? GET calls? | |
| 18:33:38 | sdague | tidwellr: is it immediately after a POST? | |
| 18:35:07 | tidwellr | sdague: no, we're troublshooting this call against an instance that's been running for a while | |