| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2018-08-22 | |||
| 12:35:42 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: [WIP] Add scatter_gather_single_cell utility https://review.openstack.org/594947 | |
| 12:37:22 | openstackgerrit | Surya Seetharaman proposed openstack/nova master: [WIP] Add scatter_gather_single_cell utility https://review.openstack.org/594947 | |
| 12:49:51 | mriedem | sahid: i see you asked for the reno on https://review.openstack.org/#/c/589548/ - do we really need a release note for this? | |
| 12:50:48 | sean-k-mooney | mriedem: i had not i was away having lunch. ill take a look now | |
| 12:51:56 | dr_gogeta86 | hi guys | |
| 12:52:07 | dr_gogeta86 | who can help me with this nova-manage problem | |
| 12:52:18 | dr_gogeta86 | says doesn't have the grant for nova_api database | |
| 12:52:29 | dr_gogeta86 | but on the other two controllers is it ok | |
| 12:52:33 | dr_gogeta86 | with the same nova.conf | |
| 12:53:07 | tssurya | dr_gogeta86: are you sure the [api_database] is set on the nova.conf you are trying ? | |
| 12:53:30 | dr_gogeta86 | yep | |
| 12:53:39 | dr_gogeta86 | and is the same along controllers | |
| 12:53:56 | tssurya | what is the nova-manage command that you are running ? | |
| 12:54:00 | dr_gogeta86 | but the error says ... it tries without password | |
| 12:54:11 | dr_gogeta86 | /usr/bin/nova-manage --log-file /tmp/test_verbose --verbose api_db sync | |
| 12:54:43 | tssurya | ah auth issues..? | |
| 12:55:05 | tssurya | doesn't look like a "nova-manage problem" not sure then.. | |
| 12:56:32 | sahid | mriedem: that seemed good to have it, i don't have strong opinion, you probably have better of view than me in such topic | |
| 12:56:56 | mriedem | we can leave it, frickler also asked for one | |
| 12:57:06 | dr_gogeta86 | tssurya, i dunno | |
| 12:58:29 | dr_gogeta86 | write(1, "error: (pymysql.err.OperationalE"..., 115error: (pymysql.err.OperationalError) (1045, u"Access denied for user 'nova'@'10.20.105.31' (using password: NO)") | |
| 12:58:29 | dr_gogeta86 | even on nova | |
| 12:58:39 | mriedem | dr_gogeta86: same user trying to run nova-manage on all the systems with the same permissions? | |
| 12:58:50 | mriedem | compare the nova user permissions | |
| 12:58:54 | mriedem | or whatever you're using to run the command | |
| 12:59:09 | dr_gogeta86 | all nodes got the same nova.conf | |
| 12:59:19 | mriedem | i'm not talking about nova.conf | |
| 12:59:25 | dr_gogeta86 | how to check it ? | |
| 13:01:45 | mriedem | are you able to run other nova-manage commands? | |
| 13:02:47 | openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: Always escape IPv6 addresses when used in migration URI https://review.openstack.org/589548 | |
| 13:03:45 | dr_gogeta86 | mriedem, no | |
| 13:03:55 | dr_gogeta86 | but i dunno why happen just in one node | |
| 13:04:02 | dr_gogeta86 | i've also checked the grant is there | |
| 13:05:46 | tssurya | mriedem: any known gate issues or do we just recheck? http://logs.openstack.org/50/594050/6/gate/tempest-slow/fe8c5c0/job-output.txt.gz#_2018-08-22_11_44_24_316767, http://logs.openstack.org/50/594050/6/gate/tempest-slow/fe8c5c0/job-output.txt.gz#_2018-08-22_11_48_21_890678 | |
| 13:06:02 | alex_xu | mriedem: yes, gmann is on vacation | |
| 13:07:18 | klosp | hi, does anyone have tips about the best way/docs to start learn how nova "works" internally? My goal in principle is not to commit anything, but I am working in a proof of concept where I would use OpenStack/nova to program an FPGA instead of booting VMs | |
| 13:10:17 | dr_gogeta86 | mriedem, i've strace it doesn't read nova.conf | |
| 13:13:08 | mriedem | tssurya: recheck | |
| 13:13:34 | mriedem | klosp: you probably want to talk to the cyborg team | |
| 13:18:17 | openstackgerrit | Merged openstack/nova stable/rocky: Update UPPER_CONSTRAINTS_FILE for stable/rocky https://review.openstack.org/590731 | |
| 13:20:49 | klosp | mriedem: thanks for the tip. I am aware of cyborg project but I am looking for some simpler and faster solution for demo only. In my understanding cyborg need driver support also from vendor and I am not aware that the FPGA vendor I am using will support it in the short term | |
| 13:21:18 | openstackgerrit | Merged openstack/osc-placement stable/rocky: Update .gitreview for stable/rocky https://review.openstack.org/589773 | |
| 13:22:26 | dr_gogeta86 | if i allow a grant on mysql without password it works just on one node | |
| 13:22:32 | dr_gogeta86 | and doesn't work in the others | |
| 13:27:26 | openstackgerrit | Merged openstack/osc-placement stable/rocky: Update UPPER_CONSTRAINTS_FILE for stable/rocky https://review.openstack.org/589774 | |
| 13:29:36 | dr_gogeta86 | nova-manage where did he get db settings ? | |
| 13:29:44 | dr_gogeta86 | directly from nova.conf | |
| 13:29:48 | dr_gogeta86 | or some other service ? | |
| 13:44:15 | mriedem | dr_gogeta86: nova.conf | |
| 13:44:27 | dr_gogeta86 | doesn't fit | |
| 13:44:29 | mriedem | sean-k-mooney: is there anything bad that can happen if we are plugging an inactive port? | |
| 13:44:42 | dr_gogeta86 | i've replaced a strange value as address | |
| 13:44:50 | dr_gogeta86 | doesn't change in what i see with strace | |
| 13:45:18 | sean-k-mooney | mriedem: i am not sure. it would depend on the backend i think. | |
| 13:45:38 | mriedem | ovs | |
| 13:45:49 | mriedem | Successfully plugged vif VIFOpenVSwitch(active=False,address=fa:16:3e:2b:a9:39,bridge_name='br-int',has_traffic_filtering=True,id=b835ff51-0262-4237-ab52-7b08206bd099,network=Network(6c9a79d8-b508-4a97-a15d-bf4acee8b4a8),plugin='ovs',port_profile=VIFPortProfileOpenVSwitch,preserve_on_delete=False,vif_name='tapb835ff51-02') | |
| 13:45:52 | sean-k-mooney | mriedem: for linux bridge we are just using brctl to say when a netdev with this name appears attach it to this bridge | |
| 13:46:35 | sean-k-mooney | for ovs its similar but depends on if its hybridg plug true/false or vhost-user | |
| 13:46:40 | sean-k-mooney | vhost user is safe | |
| 13:47:15 | mriedem | this is the gate so not vhostuser | |
| 13:47:28 | mriedem | "ovs_hybrid_plug": false | |
| 13:47:29 | sean-k-mooney | hybridge-plug=false is the same as linux bridge. we are just adding a port to ovs for a netdev that currently does not exists. ovs will wire it up when it gets created | |
| 13:49:25 | sean-k-mooney | hybrid-plug=true is less safe. we are adding a veth pair to ovs and adding the other end to a linux bridge. traffic can flow form ovs to the linux bridge and back but it cant leave the linux bridge and enter the host. | |
| 13:49:27 | openstack | Launchpad bug 1788403 in OpenStack Compute (nova) "test_server_connectivity_cold_migration_revert randomly fails ssh check" [Medium,Confirmed] | |
| 13:49:27 | mriedem | ok was just trying to debug a gate failure with resize revert https://bugs.launchpad.net/nova/+bug/1788403 | |
| 13:49:41 | mriedem | this test does a cold migratoin and then reverts, | |
| 13:49:50 | mriedem | then tries to ssh into the guest which intermittently fails | |
| 13:50:07 | mriedem | but tempest asserts the floating IP is active before trying to ssh | |
| 13:51:16 | sean-k-mooney | mriedem: my gess would be mac learning gets messed up because of RARP packets emited by qemu as part of the migrate. i would have to check however. | |
| 13:51:37 | sean-k-mooney | pluging the deactivated port should not cause issues however | |
| 13:53:05 | mriedem | ok | |
| 13:53:25 | mriedem | we also aren't waiting for any vif unplug/plug events before saying the server is active, | |
| 13:53:30 | mriedem | but that seems to be on purpose | |
| 13:54:11 | mriedem | https://review.openstack.org/#/c/179228/ | |
| 13:55:16 | sean-k-mooney | did you repoduce this locally by the way. it would be interesting to see if the vm actully had an ip when it failed. i can try to repoduce after i test your live migration lifcycled event fixes if that would help | |
| 13:55:44 | mriedem | no this is something i've seen failing in the gate | |
| 13:56:58 | sean-k-mooney | the gate job was just default ovs live migration env | |
| 13:57:10 | mriedem | this isn't live migration | |
| 13:57:11 | mriedem | cold migrate | |
| 13:57:18 | mriedem | it's also totally intermittent | |
| 13:57:22 | sean-k-mooney | sorry yes that is what i ment | |
| 13:57:43 | sean-k-mooney | its just a default ovs multnode devstack install. | |
| 13:59:22 | dr_gogeta86 | mriedem, CONF = nova.conf.CONF what is this ? | |
| 13:59:33 | mriedem | it's a global | |
| 13:59:50 | mriedem | variable that contains the conf information | |
| 14:01:01 | sean-k-mooney | dr_gogeta86: its just creating an alias to the global config object that was populated when the config was read on start up as mriedem said | |
| 14:01:26 | dr_gogeta86 | i'm wandering why doesn't read the correct one | |
| 14:01:33 | dr_gogeta86 | i dunno which one at this point | |
| 14:03:19 | sean-k-mooney | dr_gogeta86: you can spcify the config to read with --config-file or --config-dir e.g. nova-manage --config-dir /etc/nova or nova-manage --config-file /etc/nova/nova.conf | |
| 14:03:31 | sean-k-mooney | dr_gogeta86: that should be the default behavior however | |
| 14:03:36 | sean-k-mooney | so you should not need too | |
| 14:05:20 | dr_gogeta86 | i need to | |
| 14:05:32 | dr_gogeta86 | cause if put --config-file /etc/nova/nova.conf | |
| 14:05:36 | dr_gogeta86 | works like a charm | |
| 14:06:03 | sean-k-mooney | let me check the code to see what the default path actully is | |
| 14:06:10 | dr_gogeta86 | tnx you | |
| 14:08:39 | s10 | Is there any way to specify custom logging config for nova-api running under uwsgi without using separate nova.conf for nova-api/nova-scheduler/nova-conductor? For simple nova-api we can pass --log-config-append option, but for uwsgi it seems not possible. | |
| 14:11:38 | openstackgerrit | Dan Smith proposed openstack/nova master: Make RecordWrapper record RequestContext and expose cell_uuid https://review.openstack.org/594572 | |
| 14:12:05 | dansmith | yikun_: Kevin_Zheng tssurya: fixed that one thing in the tests ^ | |
| 14:12:42 | dansmith | I have suddenly collected a "cells performance mafia" following | |