| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-06-20 | |||
| 15:21:56 | jaypipes | knikolla_phone: yep | |
| 15:23:18 | knikolla_phone | jaypipes: okay, i see a deleted row. Going to try setting that too. | |
| 15:23:32 | knikolla_phone | Thanks for all your help. | |
| 15:23:36 | mnaser | i've been doing some digging around and i cannot seem to find where root_device_name gets assigned for vms.. can someone give a hint :( | |
| 15:23:42 | jaypipes | knikolla_phone: hey, no prob :) | |
| 15:23:53 | mnaser | i've discovered a bug with when using virtio-scsi with nova | |
| 15:24:06 | jaypipes | mnaser: depends on the hypervisor. | |
| 15:24:13 | jaypipes | mnaser: you talking about libvirt? | |
| 15:24:15 | mnaser | root_device_name gets set to /dev/vda .. but when attaching cinder volumes, it tries to attach them at /dev/sdX, and tries to assign the same pci address | |
| 15:24:16 | mnaser | yep | |
| 15:24:38 | jaypipes | mnaser: hmm... | |
| 15:24:49 | jaypipes | mnaser: might need to bring in mdbooth on that one. | |
| 15:24:51 | mnaser | i turned off the vm, attached the cinder block device and then started it | |
| 15:24:55 | jaypipes | mdbooth: Matt, any ideas? | |
| 15:24:57 | mnaser | and it actually generated invalid libvirt xml | |
| 15:25:01 | mnaser | i have a copy of it, let me paste | |
| 15:26:04 | mnaser | well looks like paste.openstack.org isn't loading | |
| 15:26:11 | mriedem | mnaser: _validate_and_build_base_options in nova.compute.api.API | |
| 15:26:26 | mnaser | https://gist.github.com/mnaser/8b000d20590e4343e4d7a73459d85210 | |
| 15:26:31 | mriedem | sets it in the base_options dict which eventually gets set on the instance (really the BuildRequest now) | |
| 15:26:42 | mnaser | notice the address of the first 2 disks | |
| 15:26:44 | mnaser | is the same | |
| 15:26:47 | mnaser | so qemu never boots the vm | |
| 15:27:29 | mnaser | i added a LOG.error(disk_info) in the libvirt tooling | |
| 15:27:30 | mnaser | and this is what came out of it too - https://gist.github.com/mnaser/aedd44b9df018786a144f7c719ec5a13 | |
| 15:27:49 | mnaser | notice how 'dev': 'vda' (which is ephermal) and the 'dev': 'sda' (which is attached cinder block device) | |
| 15:28:11 | mnaser | i am guessing that's where the issue is stemming, the first one should be sda too (because using virtio-scsi with the image hw_ parameters) | |
| 15:28:12 | mriedem | mnaser: https://github.com/openstack/nova/blob/9045a98145f0834446d668e0282ddb1c6f2d136c/nova/compute/api.py#L921 is eventually what gets set in the instance.root_device_name field | |
| 15:29:22 | mnaser | i've confirmed with two different installs that booting an image with hw_scsi_model=virtio-scsi and hw_disk_bus=scsi will work but no cinder volumes will successfully attach: libvirtError: internal error: unable to execute QEMU command '__com.redhat_drive_add': Duplicate ID 'drive-scsi0-0-0-0' for drive | |
| 15:29:30 | mdbooth | mnaser: what's the symptom of the weird xml? | |
| 15:29:35 | mdbooth | Does it barf? | |
| 15:29:50 | mnaser | so, if you try to attach it to an already running vm, you get the libvirt error above (duplicate id) | |
| 15:30:19 | mnaser | if you turn the vm off, attach (really a no-op if its offline), start it, it'll generate that invalid xml and fail | |
| 15:30:32 | mnaser | so im not sure if libvirt or nova is to blame in putting the duplicate address ther | |
| 15:30:45 | mdbooth | We really shouldn't be looking at root_device_name | |
| 15:31:34 | mnaser | i've just been kinda trying to understand the code and i dont know if i can point the finger at nova or libvirt yet | |
| 15:32:03 | mnaser | but it's obviously just an issue when using virtio-scsi | |
| 15:37:48 | mgagne | mriedem: supports_volume_extend vs supports_extend_volume ? RE https://review.openstack.org/#/c/454322/18/nova/virt/driver.py | |
| 15:38:31 | openstackgerrit | Dan Smith proposed openstack/nova master: Fix using targeted cell context when finding services in cells https://review.openstack.org/475830 | |
| 15:38:53 | dansmith | dtantsur: mriedem ^ | |
| 15:39:01 | dtantsur | nice | |
| 15:40:09 | openstackgerrit | Kaitlin Farr proposed openstack/nova master: Remove deprecated keymgr code https://review.openstack.org/439855 | |
| 15:41:04 | mriedem | melwitt: i think you might have gotten delete happy in the tests here https://review.openstack.org/#/c/457862/ | |
| 15:41:33 | mriedem | mgagne: i'm ok with either - i suppose supports_extend_volume to match the driver's method name | |
| 15:41:49 | mgagne | that's my opinion | |
| 15:50:22 | mriedem | mdbooth: i think that root_device_name thing is a legacy bdm v1 thing, but not totally sure | |
| 15:50:44 | mriedem | some kind of ec2 carry over | |
| 15:51:36 | mdbooth | mriedem: I haven't followed the code all the way down, but I suspect we could ignore it | |
| 15:54:36 | openstackgerrit | Jay Pipes proposed openstack/nova master: placement: scheduler uses allocation candidates https://review.openstack.org/475600 | |
| 15:56:50 | dansmith | jaypipes: quick +W bait: https://review.openstack.org/#/c/475830/ | |
| 15:57:01 | dansmith | jaypipes: fixes the ironic people | |
| 15:57:31 | jaypipes | dansmith: wallaby'd | |
| 15:57:38 | dansmith | thanks | |
| 15:57:41 | dtantsur | thanks all! | |
| 15:57:51 | dansmith | I guess cdent still isn't around today? | |
| 15:58:06 | jaypipes | dansmith: he's on PTO for next two weeksd. | |
| 15:58:11 | dansmith | oh my | |
| 15:58:20 | jaypipes | dansmith: after which he will be a VMWare employee. | |
| 15:59:29 | dansmith | oh | |
| 15:59:32 | dansmith | my | |
| 15:59:35 | dansmith | GAWD | |
| 16:01:34 | openstackgerrit | Merged openstack/nova master: Count networks to check quota https://review.openstack.org/446241 | |
| 16:02:29 | gibi_ | jaypipes: good luck with the api.fault bug | |
| 16:02:46 | jaypipes | gibi_: yeah, I might need to pass on that :( | |
| 16:02:56 | jaypipes | gibi_: that's some old-ass code in there :) | |
| 16:03:13 | gibi_ | jaypipes: I hope somebody can say 'delete that code!' | |
| 16:04:47 | jaypipes | gibi_: it's possible. some day. https://twitter.com/dims/status/542002018970529792 | |
| 16:06:27 | gibi_ | :) | |
| 16:07:36 | openstackgerrit | Merged openstack/nova master: Add SecurityGroupList.get_counts() https://review.openstack.org/446245 | |
| 16:08:08 | openstackgerrit | Merged openstack/nova master: Count security groups to check quota https://review.openstack.org/457860 | |
| 16:09:12 | openstackgerrit | Merged openstack/nova master: Add FixedIPList.get_count_by_project() https://review.openstack.org/446246 | |
| 16:09:48 | openstackgerrit | Merged openstack/nova master: Count fixed ips to check quota https://review.openstack.org/457861 | |
| 16:10:17 | openstackgerrit | Merged openstack/nova master: Add FloatingIPList.get_count_by_project() https://review.openstack.org/446247 | |
| 16:10:23 | dansmith | wow, just count those counting quotas merges | |
| 16:21:09 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Factor out duplicated notification sample data (3) https://review.openstack.org/452820 | |
| 16:21:09 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Factor out duplicated notification sample data https://review.openstack.org/452818 | |
| 16:21:10 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Factor out duplicated notification sample data (2) https://review.openstack.org/452819 | |
| 16:21:10 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: Factor out duplicated notification sample data https://review.openstack.org/475860 | |
| 16:25:34 | openstackgerrit | Gábor Antal proposed openstack/nova master: Transform keypair.delete notification https://review.openstack.org/463946 | |
| 16:32:01 | openstackgerrit | Alex Szarka proposed openstack/nova master: [WIP]Transform instance-live_migration_abort notification https://review.openstack.org/420453 | |
| 16:36:15 | openstackgerrit | Alex Szarka proposed openstack/nova master: [WIP]Transform instance-live_migration_abort notification https://review.openstack.org/420453 | |
| 16:37:55 | openstackgerrit | Alex Szarka proposed openstack/nova master: [WIP]Transform instance-live_migration_abort notification https://review.openstack.org/420453 | |
| 16:40:06 | openstackgerrit | Alex Szarka proposed openstack/nova master: [WIP]Transform instance-live_migration_abort notification https://review.openstack.org/420453 | |
| 16:47:24 | openstackgerrit | Gábor Antal proposed openstack/nova master: Add json style checking for sample notifications https://review.openstack.org/443677 | |
| 17:22:28 | openstackgerrit | Merged openstack/nova master: Updated from global requirements https://review.openstack.org/475563 | |
| 18:26:59 | mriedem | gibi_: i know this was already approved, but needed to be rebased and while it's sitting there i had a question inline https://review.openstack.org/#/c/460029/8/doc/notification_samples/instance-soft_delete-start.json | |
| 18:27:07 | mriedem | about transforming soft delete notifications to versioned payloads | |
| 18:36:44 | melwitt | mriedem: I deleted those tests on purpose because after changing floating_ips to a CountableResource, there are no more "project scoped quotas" that are refreshable or syncable. the majority of the tests in nova/tests/unit/test_quota.py rely on the global QUOTAS engine and once there are no more real examples of a certain type of quota, we would have to synthesize fake resources for the tests to still work | |
| 18:37:51 | melwitt | instead, I figured delete them since they're no longer applicable to anything real. the quota refresh command only refreshes ReservableResources | |
| 18:38:42 | mriedem | i thought the refresh command was using the _sync_*_quotas stuff still, which is in the db api, but would have to dig | |
| 18:38:57 | mriedem | if it's only using ReservableResources, and we don't have any of those, that's also grounds to remove it - but i need to check | |
| 18:39:27 | melwitt | mriedem: it does, but the _sync_* functions are only used on ReservableResources. the connection is made in nova/quota.py and each patch that removes the connection stops it from working that way | |
| 18:39:55 | melwitt | mriedem: we still have ReservableResources, they are just only user scoped and those unit tests are testing project scoped. it's confusing :( | |
| 18:39:57 | openstackgerrit | Kaitlin Farr proposed openstack/nova master: Remove deprecated keymgr code https://review.openstack.org/439855 | |
| 18:40:44 | mriedem | i'll dig a bit | |
| 18:42:15 | melwitt | the patch that removes the last ReservableResource is "Count instances" and we're not there yet. k. the "user scoped quotas" are the ones in the "project_user_quotas_table" and the "project scoped quotas" are in the "quotas" table (floating_ips, fixed_ips, networks). that's what the tests refer to when it says "project key" it means one of: 'floating_ips', 'fixed_ips', or 'networks' | |
| 18:43:12 | mriedem | melwitt: ok so https://github.com/openstack/nova/blob/master/nova/quota.py#L900 checks https://github.com/openstack/nova/blob/master/nova/quota.py#L416 | |
| 18:43:15 | melwitt | there's also the global var in nova/db/sqlalchemy/api.py PER_PROJECT_QUOTAS that show the project keys | |
| 18:43:38 | mriedem | and since per_project_resources is fixedips, floatingips and networks, | |
| 18:43:48 | mriedem | and none of those are ReservableResources | |