| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-08-22 | |||
| 13:49:13 | bhagyashris | dansmith, jaypipes, melwitt: Hi, I want to discuss regarding bfv case, I have done some testing on the stable/pike and what i observed is even if I have boot the instance from volume then in that case it allocates the disk_gb as the flavor root_gb in both compute_nodes and allocations table but after applying the patch [1]: https://review.openstack.org/#/c/428481/ on stable/pike then it makes the correct allocations. So why these two patche | |
| 13:49:14 | bhagyashris | [1][2] are ignored in the pike. [1]: https://review.openstack.org/#/c/428481/ [2]: https://review.openstack.org/#/c/428505/ | |
| 13:49:32 | lajoskatona | :ew | |
| 13:49:34 | lajoskatona | :w | |
| 14:13:02 | openstackgerrit | Merged openstack/nova master: Add functional force live migrate test https://review.openstack.org/495170 | |
| 14:15:05 | openstackgerrit | Merged openstack/nova master: Add functional live migrate test https://review.openstack.org/495811 | |
| 14:35:42 | mriedem | dansmith: we should probably re-order the stack here https://review.openstack.org/#/c/496032/ so that yours goes first as it's basically ready and mine is kind of a mess | |
| 14:36:20 | dansmith | mriedem: we can't because mine will fail if the allocation doesn't get doubled | |
| 14:36:36 | dansmith | mriedem: are you working on those unit test failures? | |
| 14:37:32 | mriedem | dansmith: my change can't go in like this, it changes the api behavior on that force flag | |
| 14:37:42 | mriedem | so i likely have to pass a skip_filters flag to select_destinations now | |
| 14:37:59 | mriedem | or have conductor create the allocations for the dest, which we talked about not wanting to do | |
| 14:38:07 | dansmith | mriedem: I missed the discussion on what this changes | |
| 14:38:13 | mriedem | hangout? | |
| 14:38:16 | dansmith | okay | |
| 14:38:58 | dansmith | mriedem: https://hangouts.google.com/call/ouqrk7fijja6pkeeackdtnuc7iu | |
| 14:49:42 | tobasco | interesting, this patch was merged 2 years and 9 months ago and still exists in master https://review.openstack.org/#/c/109537/ | |
| 14:49:56 | tobasco | so you cannot boot vm:s from cinder volumes using vda since then | |
| 14:50:34 | tobasco | hitting this when testing upgrade between liberty->mitaka | |
| 14:57:50 | openstackgerrit | Elod Illes proposed openstack/nova master: Cover migration fail case with functional test https://review.openstack.org/496280 | |
| 15:02:33 | smcginnis | tobasco: Still hitting that? Someone else in the cinder channel yesterday was having trouble with that too. | |
| 15:03:30 | smcginnis | tobasco: This worked for them for the first attempt, then they got DB errors after that: https://review.openstack.org/#/c/428481/ | |
| 15:04:45 | tobasco | smcginnis: hm is that really related thought | |
| 15:05:28 | tobasco | im comparing and trying to figure out why it check source_type in mitaka and destination_type in master for the BDM, I assume that part has been severely changed. | |
| 15:05:33 | smcginnis | tobasco: Oh, probably not. Sorry, I scanned that too quickly. | |
| 15:05:50 | tobasco | smcginnis: thanks thought :) | |
| 15:06:28 | smcginnis | tobasco: You had the other issue in the past though, right? With Cinder volumes getting counted against local space. | |
| 15:07:09 | tobasco | smcginnis: yeah, that's true we are working on moving as many versions as possible now, this is the first hop | |
| 15:07:37 | tobasco | a lot of compatibility checking right now, and was no release notes on the above "force" to vdb mounting volumes as to vda when booting volume as root volume in liberty | |
| 15:07:50 | tobasco | *compared to in liberty | |
| 15:08:08 | smcginnis | tobasco: Ah, can't really help you there, but I'm sure someone else here can. Good luck! | |
| 15:09:40 | tobasco | smcginnis: thanks | |
| 15:10:30 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Remove some single use function from utils https://review.openstack.org/491513 | |
| 15:23:07 | openstackgerrit | Stephen Finucane proposed openstack/nova master: rbd: Remove unnecessary 'encode' calls https://review.openstack.org/412356 | |
| 15:28:29 | tobasco | to answer my own question above, you can boot with a volume as root disk using vda if you the root_device_name on the image you are booting to something else and therefore tricking it | |
| 15:28:33 | tobasco | glance image-update --property root_device_name=fake db08860c-84de-4096-a52f-a32236396404 | |
| 15:28:51 | tobasco | ugly fix, but this should have been in the release notes, and even perhaps a way to disable the behaviour | |
| 15:29:16 | tobasco | for anybody finding this when searching, i haven't tested this other than with our own very specific use-case | |
| 15:30:23 | stephenfin | mdbooth: Does this make sense? https://review.openstack.org/#/c/491589/ | |
| 15:43:04 | cfriesen_ | tobasco: isn't the patch you pointed to intended to cover the case where you are doing boot-from-image with a separate volume specified (where the volume isn't the boot disk) | |
| 15:48:24 | tobasco | cfriesen_: unsure, it does apply when just doing the "boot from image (create new volume)" way | |
| 15:48:58 | tobasco | we only do pure cinder backed vm:s with a cinder volume as root disk, setting the root_device_name to something other than vda allowed us to boot the volume as vda. | |
| 15:49:33 | tobasco | note that the ubuntu 16.04 vm interpreted it as one disk and mapped it to vda but openstack said it was vda and set it to "disk 1" instead of 0 in the XML on the kvm node | |
| 16:10:44 | mriedem | dansmith: in the case that claim_resources fails for the dest node, what are you thinking we do? warn and continue to force or fail with NoValidHost? | |
| 16:10:54 | mriedem | s/NoValidHost/MigrationPreCheckError/ | |
| 16:10:58 | cfriesen_ | tobasco: sorry, was testing this. for what it's worth, in our Mitaka setup with libvirt/qemu/ceph-based-volumes in the "boot from image (create new volume)" case I get the volume on /dev/vda with the root filesystem on /dev/vda1. | |
| 16:11:31 | mriedem | dansmith: i'm thinking we fail since if the RT isn't going to heal the allocations, we really don't want that missing being claimed for the dest host, | |
| 16:11:44 | openstackgerrit | Merged openstack/nova master: Add description on maximum placement API version https://review.openstack.org/492056 | |
| 16:11:45 | mriedem | since it would screw up scheduling decisions for other instances on that node later | |
| 16:12:00 | dansmith | mriedem: yeah have to fail, IMHO | |
| 16:12:53 | cfriesen_ | tobasco: this is with no ephemeral/swap configured. | |
| 16:16:29 | cfriesen_ | mriedem: dansmith: for what it's worth, I agree with failing if we can't claim resources. otherwise it'd cause an unholy mess for things like cpu pinning, PCI devices, hugepages, etc. | |
| 16:26:25 | mriedem | ok got the code fix, running the functional test now to make sure it's legit | |
| 16:41:02 | sterdnotshaken | How do you delete a network node entirely from Openstack? | |
| 16:41:15 | sterdnotshaken | Let say I have 2 compute and 2 network nodes and the hardware for one of the network nodes fails entirely... | |
| 16:41:26 | sterdnotshaken | So I just want to delete all references to that network node from openstack and create a new network node on new hardware... | |
| 16:41:59 | mriedem | sterdnotshaken: probably better asked in #openstack-operators | |
| 16:42:18 | sterdnotshaken | ok | |
| 16:45:12 | mriedem | well look who it is https://www.youtube.com/watch?v=NzWw_uosDnM&index=6&list=PLOuHvpVx7kYksG0NFaCaQsSkrUlj3Oq4S | |
| 16:50:19 | mriedem | alright functional test is passing, time to fix up the unit tests | |
| 16:52:47 | mriedem | melwitt: are you going to sign up for the red hat pike ptg video this time? | |
| 16:53:01 | mriedem | because you should | |
| 16:53:28 | mriedem | you can be like, quotas this, quotas that, something something claims in the scheduler | |
| 17:05:19 | openstackgerrit | Merged openstack/nova master: Add track_instance_changes note in disable_group_policy_check_upcall https://review.openstack.org/490627 | |
| 17:05:45 | openstackgerrit | Merged openstack/nova master: doc: code review considerations for online data migrations https://review.openstack.org/491885 | |
| 17:06:10 | openstackgerrit | Merged openstack/nova master: Fix a wrong link https://review.openstack.org/494109 | |
| 17:10:36 | openstackgerrit | Drew Fisher proposed openstack/nova master: Add language for compute node configuration https://review.openstack.org/489643 | |
| 17:35:56 | mriedem | wtf, this is super helpful https://gist.github.com/mriedem/706bcfbac773660310c38ad319ce7fe2 | |
| 17:36:11 | mriedem | doesn't exist, create it, sorry it already exists, ok give me it, sorry, it doesn't exist, dummy | |
| 17:43:08 | mriedem | oh SmallFakeDriver, you'll be the death of me | |
| 17:57:42 | ganlaksh | test | |
| 18:03:43 | clarkb | mriedem: dansmith https://docs.openstack.org/nova/latest/admin/ssh-configuration.html says that you need to configure the nova user so that nova can ssh to other compute hosts to move disks around. We don't set this up in devstack-gate/devstack multinode jobs. Is this actually required if using libvirt which ssh's as root? | |
| 18:04:41 | jlk | specifically curious about instance resize | |
| 18:05:14 | mriedem | devstack configures some stuff for live migration using ssh here https://github.com/openstack-dev/devstack/blob/master/lib/nova_plugins/hypervisor-libvirt#L45 | |
| 18:05:21 | mriedem | that option is deprecated though | |
| 18:05:37 | mriedem | you can tunnel but i believe that's discouraged | |
| 18:06:26 | mriedem | i had brought up the live_migration_uri confusion in http://lists.openstack.org/pipermail/openstack-dev/2017-June/117987.html | |
| 18:06:39 | mriedem | because i was trying to remove that from devstack | |
| 18:07:30 | clarkb | oh is it using the stack user? | |
| 18:07:34 | mriedem | yes | |
| 18:07:46 | clarkb | then why do we set up ssh for the root user? | |
| 18:07:57 | clarkb | (also I'll have to dig more to see where it set up stack user) | |
| 18:08:02 | mriedem | idk | |
| 18:16:15 | jlk | mriedem: is all of instance resize handled by the migration code paths now? | |
| 18:16:44 | mriedem | jlk: it always was | |
| 18:16:51 | jlk | mriedem: I could have sworn that in older releases (mitaka/newton) that a resize specifically did some ssh'ing around as the user running the nova process | |
| 18:17:26 | mriedem | resize has always equaled the cold migration code paths, | |
| 18:17:32 | mriedem | only difference is the flavor being the same or not | |
| 18:17:36 | jlk | but maybe that's because we had: live_migration_uri = "qemu+ssh://nova@%s/system?keyfile={{ nova.state_path }}/.ssh/id_rsa" | |
| 18:18:53 | mriedem | dansmith: tests are passing, just doing another run with everything to make sure i'm not missing some failing test | |
| 18:18:56 | clarkb | my understanding when we set this up for multinode testing wasw there isn't any reason to not just use root beacuse libvirt ~= root. Which is why we set up root ssh. But its been a while. I bet jogo remembers | |
| 18:19:39 | jlk | I think our local config was not happy about allowing incoming root ssh | |
| 18:19:41 | mriedem | i think if you tunnel via libvirt between the two nodes then that might be ok, but i don't think we're tunneling | |
| 18:19:43 | jlk | so we shunted it to nova | |
| 18:19:55 | mriedem | in devstack i mean | |
| 18:20:18 | mriedem | ganlaksh: hi, i saw your email, sorry i've been busy and haven't had a chance to respond | |
| 18:20:53 | mriedem | ganlaksh: https://docs.openstack.org/nova/latest/#for-contributors would be good to step through if you haven't yet | |
| 18:20:59 | mriedem | note that some of it might be outdated | |
| 18:22:59 | ganlaksh | mriedem: Thanks! Yes I have been going through these docs .. Will continue to look into it.. | |
| 18:27:04 | mriedem | ganlaksh: are you interested in working on a specific part of nova? | |