| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-08-22 | |||
| 14:37:42 | mriedem | so i likely have to pass a skip_filters flag to select_destinations now | |
| 14:37:59 | mriedem | or have conductor create the allocations for the dest, which we talked about not wanting to do | |
| 14:38:07 | dansmith | mriedem: I missed the discussion on what this changes | |
| 14:38:13 | mriedem | hangout? | |
| 14:38:16 | dansmith | okay | |
| 14:38:58 | dansmith | mriedem: https://hangouts.google.com/call/ouqrk7fijja6pkeeackdtnuc7iu | |
| 14:49:42 | tobasco | interesting, this patch was merged 2 years and 9 months ago and still exists in master https://review.openstack.org/#/c/109537/ | |
| 14:49:56 | tobasco | so you cannot boot vm:s from cinder volumes using vda since then | |
| 14:50:34 | tobasco | hitting this when testing upgrade between liberty->mitaka | |
| 14:57:50 | openstackgerrit | Elod Illes proposed openstack/nova master: Cover migration fail case with functional test https://review.openstack.org/496280 | |
| 15:02:33 | smcginnis | tobasco: Still hitting that? Someone else in the cinder channel yesterday was having trouble with that too. | |
| 15:03:30 | smcginnis | tobasco: This worked for them for the first attempt, then they got DB errors after that: https://review.openstack.org/#/c/428481/ | |
| 15:04:45 | tobasco | smcginnis: hm is that really related thought | |
| 15:05:28 | tobasco | im comparing and trying to figure out why it check source_type in mitaka and destination_type in master for the BDM, I assume that part has been severely changed. | |
| 15:05:33 | smcginnis | tobasco: Oh, probably not. Sorry, I scanned that too quickly. | |
| 15:05:50 | tobasco | smcginnis: thanks thought :) | |
| 15:06:28 | smcginnis | tobasco: You had the other issue in the past though, right? With Cinder volumes getting counted against local space. | |
| 15:07:09 | tobasco | smcginnis: yeah, that's true we are working on moving as many versions as possible now, this is the first hop | |
| 15:07:37 | tobasco | a lot of compatibility checking right now, and was no release notes on the above "force" to vdb mounting volumes as to vda when booting volume as root volume in liberty | |
| 15:07:50 | tobasco | *compared to in liberty | |
| 15:08:08 | smcginnis | tobasco: Ah, can't really help you there, but I'm sure someone else here can. Good luck! | |
| 15:09:40 | tobasco | smcginnis: thanks | |
| 15:10:30 | openstackgerrit | Stephen Finucane proposed openstack/nova master: trivial: Remove some single use function from utils https://review.openstack.org/491513 | |
| 15:23:07 | openstackgerrit | Stephen Finucane proposed openstack/nova master: rbd: Remove unnecessary 'encode' calls https://review.openstack.org/412356 | |
| 15:28:29 | tobasco | to answer my own question above, you can boot with a volume as root disk using vda if you the root_device_name on the image you are booting to something else and therefore tricking it | |
| 15:28:33 | tobasco | glance image-update --property root_device_name=fake db08860c-84de-4096-a52f-a32236396404 | |
| 15:28:51 | tobasco | ugly fix, but this should have been in the release notes, and even perhaps a way to disable the behaviour | |
| 15:29:16 | tobasco | for anybody finding this when searching, i haven't tested this other than with our own very specific use-case | |
| 15:30:23 | stephenfin | mdbooth: Does this make sense? https://review.openstack.org/#/c/491589/ | |
| 15:43:04 | cfriesen_ | tobasco: isn't the patch you pointed to intended to cover the case where you are doing boot-from-image with a separate volume specified (where the volume isn't the boot disk) | |
| 15:48:24 | tobasco | cfriesen_: unsure, it does apply when just doing the "boot from image (create new volume)" way | |
| 15:48:58 | tobasco | we only do pure cinder backed vm:s with a cinder volume as root disk, setting the root_device_name to something other than vda allowed us to boot the volume as vda. | |
| 15:49:33 | tobasco | note that the ubuntu 16.04 vm interpreted it as one disk and mapped it to vda but openstack said it was vda and set it to "disk 1" instead of 0 in the XML on the kvm node | |
| 16:10:44 | mriedem | dansmith: in the case that claim_resources fails for the dest node, what are you thinking we do? warn and continue to force or fail with NoValidHost? | |
| 16:10:54 | mriedem | s/NoValidHost/MigrationPreCheckError/ | |
| 16:10:58 | cfriesen_ | tobasco: sorry, was testing this. for what it's worth, in our Mitaka setup with libvirt/qemu/ceph-based-volumes in the "boot from image (create new volume)" case I get the volume on /dev/vda with the root filesystem on /dev/vda1. | |
| 16:11:31 | mriedem | dansmith: i'm thinking we fail since if the RT isn't going to heal the allocations, we really don't want that missing being claimed for the dest host, | |
| 16:11:44 | openstackgerrit | Merged openstack/nova master: Add description on maximum placement API version https://review.openstack.org/492056 | |
| 16:11:45 | mriedem | since it would screw up scheduling decisions for other instances on that node later | |
| 16:12:00 | dansmith | mriedem: yeah have to fail, IMHO | |
| 16:12:53 | cfriesen_ | tobasco: this is with no ephemeral/swap configured. | |
| 16:16:29 | cfriesen_ | mriedem: dansmith: for what it's worth, I agree with failing if we can't claim resources. otherwise it'd cause an unholy mess for things like cpu pinning, PCI devices, hugepages, etc. | |
| 16:26:25 | mriedem | ok got the code fix, running the functional test now to make sure it's legit | |
| 16:41:02 | sterdnotshaken | How do you delete a network node entirely from Openstack? | |
| 16:41:15 | sterdnotshaken | Let say I have 2 compute and 2 network nodes and the hardware for one of the network nodes fails entirely... | |
| 16:41:26 | sterdnotshaken | So I just want to delete all references to that network node from openstack and create a new network node on new hardware... | |
| 16:41:59 | mriedem | sterdnotshaken: probably better asked in #openstack-operators | |
| 16:42:18 | sterdnotshaken | ok | |
| 16:45:12 | mriedem | well look who it is https://www.youtube.com/watch?v=NzWw_uosDnM&index=6&list=PLOuHvpVx7kYksG0NFaCaQsSkrUlj3Oq4S | |
| 16:50:19 | mriedem | alright functional test is passing, time to fix up the unit tests | |
| 16:52:47 | mriedem | melwitt: are you going to sign up for the red hat pike ptg video this time? | |
| 16:53:01 | mriedem | because you should | |
| 16:53:28 | mriedem | you can be like, quotas this, quotas that, something something claims in the scheduler | |
| 17:05:19 | openstackgerrit | Merged openstack/nova master: Add track_instance_changes note in disable_group_policy_check_upcall https://review.openstack.org/490627 | |
| 17:05:45 | openstackgerrit | Merged openstack/nova master: doc: code review considerations for online data migrations https://review.openstack.org/491885 | |
| 17:06:10 | openstackgerrit | Merged openstack/nova master: Fix a wrong link https://review.openstack.org/494109 | |
| 17:10:36 | openstackgerrit | Drew Fisher proposed openstack/nova master: Add language for compute node configuration https://review.openstack.org/489643 | |
| 17:35:56 | mriedem | wtf, this is super helpful https://gist.github.com/mriedem/706bcfbac773660310c38ad319ce7fe2 | |
| 17:36:11 | mriedem | doesn't exist, create it, sorry it already exists, ok give me it, sorry, it doesn't exist, dummy | |
| 17:43:08 | mriedem | oh SmallFakeDriver, you'll be the death of me | |
| 17:57:42 | ganlaksh | test | |
| 18:03:43 | clarkb | mriedem: dansmith https://docs.openstack.org/nova/latest/admin/ssh-configuration.html says that you need to configure the nova user so that nova can ssh to other compute hosts to move disks around. We don't set this up in devstack-gate/devstack multinode jobs. Is this actually required if using libvirt which ssh's as root? | |
| 18:04:41 | jlk | specifically curious about instance resize | |
| 18:05:14 | mriedem | devstack configures some stuff for live migration using ssh here https://github.com/openstack-dev/devstack/blob/master/lib/nova_plugins/hypervisor-libvirt#L45 | |
| 18:05:21 | mriedem | that option is deprecated though | |
| 18:05:37 | mriedem | you can tunnel but i believe that's discouraged | |
| 18:06:26 | mriedem | i had brought up the live_migration_uri confusion in http://lists.openstack.org/pipermail/openstack-dev/2017-June/117987.html | |
| 18:06:39 | mriedem | because i was trying to remove that from devstack | |
| 18:07:30 | clarkb | oh is it using the stack user? | |
| 18:07:34 | mriedem | yes | |
| 18:07:46 | clarkb | then why do we set up ssh for the root user? | |
| 18:07:57 | clarkb | (also I'll have to dig more to see where it set up stack user) | |
| 18:08:02 | mriedem | idk | |
| 18:16:15 | jlk | mriedem: is all of instance resize handled by the migration code paths now? | |
| 18:16:44 | mriedem | jlk: it always was | |
| 18:16:51 | jlk | mriedem: I could have sworn that in older releases (mitaka/newton) that a resize specifically did some ssh'ing around as the user running the nova process | |
| 18:17:26 | mriedem | resize has always equaled the cold migration code paths, | |
| 18:17:32 | mriedem | only difference is the flavor being the same or not | |
| 18:17:36 | jlk | but maybe that's because we had: live_migration_uri = "qemu+ssh://nova@%s/system?keyfile={{ nova.state_path }}/.ssh/id_rsa" | |
| 18:18:53 | mriedem | dansmith: tests are passing, just doing another run with everything to make sure i'm not missing some failing test | |
| 18:18:56 | clarkb | my understanding when we set this up for multinode testing wasw there isn't any reason to not just use root beacuse libvirt ~= root. Which is why we set up root ssh. But its been a while. I bet jogo remembers | |
| 18:19:39 | jlk | I think our local config was not happy about allowing incoming root ssh | |
| 18:19:41 | mriedem | i think if you tunnel via libvirt between the two nodes then that might be ok, but i don't think we're tunneling | |
| 18:19:43 | jlk | so we shunted it to nova | |
| 18:19:55 | mriedem | in devstack i mean | |
| 18:20:18 | mriedem | ganlaksh: hi, i saw your email, sorry i've been busy and haven't had a chance to respond | |
| 18:20:53 | mriedem | ganlaksh: https://docs.openstack.org/nova/latest/#for-contributors would be good to step through if you haven't yet | |
| 18:20:59 | mriedem | note that some of it might be outdated | |
| 18:22:59 | ganlaksh | mriedem: Thanks! Yes I have been going through these docs .. Will continue to look into it.. | |
| 18:27:04 | mriedem | ganlaksh: are you interested in working on a specific part of nova? | |
| 18:27:31 | mriedem | also, what timezone are you in? | |
| 18:28:27 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Allocate resources on forced dest host during live migration https://review.openstack.org/496031 | |
| 18:28:27 | openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove source node allocation after live migration completes https://review.openstack.org/496032 | |
| 18:28:40 | mriedem | dansmith: cfriesen_: here we are ^ | |
| 18:34:01 | ganlaksh | mriedem: I am in the PT zone.. with respect to my areas of interest . i dont have any specific right now.. I would like to pick up some bugs and go through the process.. from the top of my mind.. cells,live-migration, Vmware would be some areas I would be interested in.. I am looking at the tracking page here for the subteams . | |
| 18:35:12 | mriedem | ok, vmware hasn't had much activity in awhile, so if you could dig through some vmware-related bugs and see if they are still valid that would be helpful | |
| 18:35:13 | ganlaksh | mriedem: https://etherpad.openstack.org/p/pike-nova-priorities-tracking. tracking page that i referenced | |
| 18:35:45 | mriedem | ganlaksh: ok that's likely stale at this point | |
| 18:35:58 | mriedem | pike is about to be released, so we'll start focusing on the next release, which is named queens | |
| 18:36:13 | mriedem | ganlaksh: here are open vmware bugs https://bugs.launchpad.net/nova/+bugs?field.tag=vmware | |