| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2019-11-12 | |||
| 17:21:35 | sean-k-mooney | i still need to fix the typos you noted in the code but thats next on my list | |
| 17:22:06 | sean-k-mooney | then i need to adress dansmith's feedback on another series :) | |
| 17:23:26 | mriedem | dansmith: are you saying the api change would just be changing the status code from 400 to 409 rather than a microversion? | |
| 17:23:37 | dansmith | mriedem: no | |
| 17:24:25 | mriedem | then i don't think a secondary api-only spec is going to be that big of a deal | |
| 17:24:43 | dansmith | cripes | |
| 17:24:45 | mriedem | "depends on blueprint x. in x we make y work on the compute. this spec enables it in the api with a new microversion" | |
| 17:25:08 | mriedem | just do whatever y'all think is best | |
| 17:25:13 | mriedem | i haven't read the current spec | |
| 17:25:24 | dansmith | mriedem: I'm trying to say I think we should separate the api change into its own small spec to keep lyarwood's focused on the disk changes | |
| 17:25:42 | mriedem | and i just said i was ok with that right? | |
| 17:25:53 | dansmith | mriedem: in isolation without his change, we can make the api change to remove the hard block and say "as of microversion X you may or may not be able to rescue BFVs" | |
| 17:26:13 | dansmith | mriedem: it wasn't clear to me | |
| 17:26:34 | dansmith | because that's what I've been suggesting for ten minutes and you seemed to be disagreeing, so I was trying to explain | |
| 17:26:39 | mriedem | so without any compute side changes, we should be able to rescue volume-backed servers today? | |
| 17:27:17 | mriedem | i'm fine with whatever | |
| 17:27:27 | mriedem | let's just stop talking about this | |
| 17:27:29 | dansmith | I'm clearly the problem here, I'll just butt out | |
| 17:34:55 | artom | donnyd, heya, what the difference between your multi-numa and multi-numa-expanded flavors? Or where can I look to find out? | |
| 17:35:14 | artom | (Also, we should talk about enabling hugepages at some point, but that can wait) | |
| 17:35:22 | sean-k-mooney | artom: the expanded one has 16G of ram instead of 8 | |
| 17:35:31 | artom | sean-k-mooney, that's it? | |
| 17:35:37 | sean-k-mooney | yep | |
| 17:35:51 | sean-k-mooney | otherwise they both have the same about of disk and cpu | |
| 17:35:58 | artom | And NUMA nodes | |
| 17:36:02 | sean-k-mooney | but have hw:numa_nodes=2 | |
| 17:36:25 | sean-k-mooney | and hw:cpu_socket=2 and hw:cpu_thread=2 | |
| 17:36:43 | artom | So the non-expanded one is fine | |
| 17:37:07 | sean-k-mooney | we shoudl try to use the non expanded ones if we can to save space but we might need the expanded one on the contoler | |
| 17:37:57 | artom | Ohh | |
| 17:37:58 | sean-k-mooney | ya i didnt want to have to super optimise the job to get it to fit intially which is one of the reason we started with the expanded lables but it should not be needed longterm | |
| 17:38:37 | artom | Let's try with non-expanded I guess first | |
| 17:38:43 | artom | And if we hit oom or something we can -expand :D | |
| 17:39:06 | sean-k-mooney | yep i think i was trying to allocte a low amout of hugepage in my job | |
| 17:39:15 | sean-k-mooney | well in the orginal version | |
| 17:40:00 | sean-k-mooney | i think we are generaly around the 5-6 G mark on used memroy on the contoler so you dont want to exceen more then about 1G of hugepages total | |
| 17:40:37 | artom | Ohh, you actually set hugepages in Ansible | |
| 17:40:51 | sean-k-mooney | yes here https://review.opendev.org/#/c/679656/13/playbooks/nfv/nfv.yaml | |
| 17:40:56 | artom | So I should probably do that instead of https://review.opendev.org/#/c/693690/ | |
| 17:41:01 | sean-k-mooney | but its too much currently for the contoler | |
| 17:41:07 | artom | Although that skip could still be useful in other envs | |
| 17:42:17 | sean-k-mooney | ya i would basically do this | |
| 17:42:19 | sean-k-mooney | hosts: controller | |
| 17:42:21 | sean-k-mooney | tasks: | |
| 17:42:23 | sean-k-mooney | - name: allocate hugepages | |
| 17:42:25 | sean-k-mooney | shell: echo 512 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages | |
| 17:42:27 | sean-k-mooney | become: yes | |
| 17:42:29 | sean-k-mooney | - hosts: subnode | |
| 17:42:31 | sean-k-mooney | tasks: | |
| 17:42:33 | sean-k-mooney | - name: allocate hugepages | |
| 17:42:35 | sean-k-mooney | shell: echo 2048 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages | |
| 17:42:37 | sean-k-mooney | become: yes | |
| 17:43:02 | sean-k-mooney | that will alocate 1G on the contoler and 4 on the compute of 2MB hugepages | |
| 17:43:12 | sean-k-mooney | that should be enough | |
| 17:43:55 | artom | Ack | |
| 17:55:47 | artom | sean-k-mooney, I think the problem with using https://zuul-ci.org/docs/zuul-jobs/general-roles.html#role-copy-build-sshkey is that it's 1-user only | |
| 17:55:57 | artom | IOW, 'tempest' can ssh to 'tempest' | |
| 17:56:13 | artom | We need 'tempest' to SSH to 'stack' or some other account that has limitless sudo powers | |
| 17:59:25 | sean-k-mooney | artom: the same key is already copied to root and stack | |
| 17:59:32 | sean-k-mooney | root is done in the pre job | |
| 17:59:43 | sean-k-mooney | stack in the muntinode playbook | |
| 18:00:17 | sean-k-mooney | so if we use it for tempest all 3 users will have the same private and public key | |
| 18:00:59 | sean-k-mooney | you wouldnt want to do that in production but it should be fine for ci | |
| 18:01:30 | artom | *facepalm* | |
| 18:01:31 | artom | I see | |
| 18:01:52 | artom | The devstack orchestrate-devstack role will put the pubkey in stack's authorized_keys for us | |
| 18:02:03 | artom | And we can put the private key in tempest's .ssh | |
| 18:02:13 | artom | In theory, anyways | |
| 18:05:53 | artom | ... do we have access to another project's roles? Not sure... | |
| 18:06:10 | donnyd | artom: the expanded flavors have more memory | |
| 18:11:45 | sean-k-mooney | artom: am we can get acess to them yes there are two ways to do i but your job inherits form the tempest job yes | |
| 18:11:52 | sean-k-mooney | if so then it will do it already | |
| 18:12:18 | sean-k-mooney | you you can just invoke the copy-build-key role twice once for stack and once for tempest | |
| 18:28:03 | artom | https://review.opendev.org/#/c/691062/57 let's see how badly stuff explodes | |
| 18:41:17 | sean-k-mooney | artom: you do not need the ssh keys in that anymore | |
| 18:41:36 | sean-k-mooney | but i think it should be ok | |
| 18:41:59 | sean-k-mooney | its runing in a sperate pool so it should be pretty fast | |
| 18:42:15 | artom | sean-k-mooney, something borked, it's retrying now | |
| 18:42:23 | artom | Heh, oh yeah, I should get rid of the keys | |
| 18:42:29 | artom | I'll wait for it to work first | |
| 18:43:00 | sean-k-mooney | the console for the job is here http://zuul.openstack.org/stream/11de9dc2d4fd4f7d931eedeea276af3e?logfile=console.log | |
| 18:43:23 | sean-k-mooney | looks like tis doing the normal multinode setup | |
| 18:44:30 | artom | That's the second attempt | |
| 18:44:40 | sean-k-mooney | yep | |
| 18:44:59 | sean-k-mooney | ill leave it open on one of my monitors and see what hapens | |
| 18:45:49 | sean-k-mooney | haha | |
| 18:45:54 | sean-k-mooney | you have a typo | |
| 18:46:36 | sean-k-mooney | artom: http://paste.openstack.org/show/785994/ | |
| 18:47:45 | sean-k-mooney | you did not include teh whitebox-compute role in the commit | |
| 18:47:52 | sean-k-mooney | or you forgot to remove that line | |
| 18:49:27 | artom | *facepalm* | |
| 18:49:37 | artom | 'git commit -a' isn't magic | |
| 18:49:53 | sean-k-mooney | it speciricly ignore untracted files | |
| 18:52:03 | artom | I know | |
| 18:52:42 | sean-k-mooney | good becaue i apparently cant spell specifcally and im not sure you woudl figure it out if you didnt :P | |
| 18:55:33 | sean-k-mooney | i need to try enabling spell check for irc again but the last time i tried that it was kind of depressing | |
| 18:57:29 | artom | *shrug* I forgive you :) | |
| 18:57:48 | artom | You have other uses besides spelling ;) | |
| 18:58:31 | sean-k-mooney | well if you want a terible way to generate passwords just ask me to spell somthing | |
| 18:59:14 | artom | lulz | |