| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-nova - 2017-03-03 | |||
| 11:51:49 | mdbooth | johnthetubaguy: Well if cinder doesn't know where its storage is, Nova should retire to the beach :) | |
| 11:52:13 | johnthetubaguy | mdbooth: true | |
| 11:52:16 | lyarwood | https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/rbd.py#L1024 & https://github.com/openstack/cinder/blob/master/cinder/volume/drivers/rbd.py#L339 | |
| 11:52:22 | lyarwood | they do the same as we do within imagebackend | |
| 11:52:32 | lyarwood | so they would have the same issues if everything changes | |
| 11:53:10 | johnthetubaguy | yeah, they need some kind of refresh trigger from us, maybe? | |
| 11:54:01 | lyarwood | no, you would need to manually update the ceph.conf as mdbooth said above at that point on the c-vol node | |
| 12:00:15 | lyarwood | ratailor: looking at hard reboot again, we don't disconnect and then reconnect volumes from the host let alone grab any updated connection_info from cinder at the moment | |
| 12:03:25 | ratailor | lyarwood, does that mean that we won't be able to resolve this issue fully by only doing hard reboot ? | |
| 12:04:12 | lyarwood | ratailor: yes | |
| 12:05:56 | ratailor | lyarwood, so what could be proper direction to look into ? Could you provide any help ? | |
| 12:06:28 | lyarwood | https://review.openstack.org/#/c/400384/ disconnects and then reconnects volumes during a hard reboot | |
| 12:06:50 | lyarwood | we could also add in another call to initialize_connection somewhere but that's pretty horrid | |
| 12:10:22 | ratailor | lyarwood, once this patch got merged, we need to hard reboot only. But before that we might need to figure out where to call initialize_connection. right ? | |
| 12:12:02 | lyarwood | ratailor: no, you still need to call initialize_connection and save the updated connection_info to the bdm before calling connect_volume again | |
| 12:14:18 | ratailor | lyarwood, Would if be better to handle this issue on restart of nova-compute ? | |
| 12:22:15 | sdague | johnthetubaguy: so I started in on a clean page for the limits discussion wrap up here - https://review.openstack.org/#/c/440815/ | |
| 12:22:59 | johnthetubaguy | sdague: cool, I should take a peak at that soon | |
| 12:23:57 | sdague | johnthetubaguy: it's definitely shooting at being higher level. I'm building out a references section to it now. I would like to discuss with you next steps once you get back from lunch | |
| 12:50:41 | sdague | I just read through your spec, have a chunk of it in my head. Now would be the time to jump on that before I lose it all :) | |
| 13:01:42 | johnthetubaguy | sdague: jumping now | |
| 13:01:54 | johnthetubaguy | sdague: higher level sounds like a top plan | |
| 13:02:03 | sdague | ok, so I just pushed an iterating | |
| 13:02:05 | sdague | iteration | |
| 13:02:26 | sdague | so if you haven't yet looked, https://review.openstack.org/#/c/440815/ | |
| 13:03:06 | sdague | why don't I give you a minute to look at that, then we can chat about it | |
| 13:03:39 | sdague | actually, lets chat about it in #openstack-dev, as we've had tons of related conversations with keystone folks over there effectively the last couple of days | |
| 13:03:48 | johnthetubaguy | sdague: oh, good point | |
| 13:14:03 | openstackgerrit | Arnaud Morin proposed openstack/nova master: Add warning on setting secure_proxy_ssl_header https://review.openstack.org/423375 | |
| 13:37:42 | AJaeger | sdague: nova's api-ref fails with sphinx 1.5 - I fixed it with https://review.openstack.org/#/c/441053/ | |
| 13:38:47 | sdague | AJaeger: thank you | |
| 13:45:46 | edleafe | migi: just starting my day. What's up? | |
| 13:46:07 | edleafe | oops | |
| 13:47:35 | migi | figleaf: Morning, I've run into one issue while invoking tests which I think you wrote but looks like it's some dependency issue on my system. It's failing and still scratching my head why, but I need to spent more time to find out: https://github.com/openstack/nova/blob/master/nova/tests/functional/db/test_resource_provider.py#L1390 | |
| 13:48:01 | migi | figleaf: actually one on line 1372 | |
| 13:48:53 | migi | figleaf: so I may ask you again once I know more | |
| 13:49:01 | migi | for some guidance | |
| 13:49:26 | figleaf | migi: What sort of error are you seeing | |
| 13:49:44 | figleaf | migi: Can you show a traceback in a paste? | |
| 13:51:43 | migi | figleaf: http://paste.openstack.org/show/601322/ | |
| 13:52:00 | migi | figleaf: but don't spent time on that, just if you know from the top of your head what can be cause | |
| 13:54:17 | figleaf | Looks like another test may not have cleaned things up - you have another 'CUSTOM_TWO' class created | |
| 13:54:31 | figleaf | migi: let me look a little more | |
| 13:54:56 | migi | figleaf: well this is what I thought and then renamed this to WHATEVER_TWO | |
| 13:54:59 | migi | figleaf: same error | |
| 13:55:45 | figleaf | migi: heh, was just running the same | |
| 13:57:06 | figleaf | migi: Can you add a line before your trace? print(e.columns) | |
| 13:57:13 | migi | figleaf: just for you to know it's on centos if I install everything in venv on other system like fedora then it's ok, so maybe something is missing in the system | |
| 13:57:52 | figleaf | migi: I can't imagine what would be different in centos | |
| 13:58:02 | figleaf | migi: but that is a data point | |
| 13:58:27 | figleaf | :1372 | |
| 13:58:35 | figleaf | doh! | |
| 13:58:45 | sahid | johnthetubaguy any chance you re-approve https://review.openstack.org/#/c/427066/ ? | |
| 13:58:47 | migi | figleaf: I even installed almost identical pip dependencies same error and running in venv in centos reproduces error, so pretty wired, will do just need to spin this env, will get some results in 15min | |
| 13:58:55 | frickler | AJaeger: is that the same error as this one? http://logs.openstack.org/60/421760/12/check/gate-nova-api-ref/cfe9915/console.html#_2017-03-03_04_25_58_124928 | |
| 13:59:22 | johnthetubaguy | sahid: is that just the same words we had last time? | |
| 13:59:53 | figleaf | migi: are you running your tests under tox? If so, have you tried deleting the .tox/ directory? That is a good first step when tests go wonky | |
| 14:00:20 | openstackgerrit | Alex Xu proposed openstack/nova-specs master: The ResourceProviderTraits API - Manage traits with ResourceProviders https://review.openstack.org/345138 | |
| 14:00:27 | migi | figleaf: yep tried that and with pdb I'm running with testr | |
| 14:00:39 | johnthetubaguy | sahid: if its just the same words, add the git commit tag for that: https://github.com/openstack/nova-specs/blame/master/README.rst#L123 and we can get that re-approved right away | |
| 14:02:54 | sahid | johnthetubaguy: ah ok will do | |
| 14:03:20 | AJaeger | frickler: yes | |
| 14:05:01 | AJaeger | frickler: unless that change introduces a new problem ;) | |
| 14:05:04 | openstackgerrit | sahid proposed openstack/nova-specs master: Libvirt driver emulator threads placement policy https://review.openstack.org/427066 | |
| 14:05:55 | sahid | johnthetubaguy: should be ok now, thanks | |
| 14:06:31 | johnthetubaguy | sahid: will try look at that soon, if its not swiped by someone else first | |
| 14:06:38 | johnthetubaguy | got my head in quotas right now | |
| 14:07:09 | sahid | ok understood | |
| 14:14:33 | mriedem | yikes gate is at nearly 18 hours | |
| 14:15:35 | frickler | mriedem: yup, would be nice if some infra-root would wake up ;) | |
| 14:15:52 | mriedem | frickler: i'm not sure what infra has to do with this? | |
| 14:16:30 | frickler | mriedem: the top gate check is failing to get some trusty-multinode job run. at least for the last 6 hours while I'm watching it | |
| 14:16:51 | mriedem | oh i see gate-grenade-dsvm-neutron-multinode-ubuntu-trusty | |
| 14:17:04 | mriedem | sdague: mordred: fungi: have you seen ^? | |
| 14:20:48 | mriedem | zuul jobs launched per hour has dropped way off, is zuul v3 being used now? | |
| 14:21:11 | mriedem | i couldn't get through the massive zuul v3 update emails | |
| 14:28:11 | AJaeger | mriedem: zuul v3 is not really used yet | |
| 14:29:02 | sahid | mriedem: any chances you remove your -2 here https://review.openstack.org/#/c/385023/ | |
| 14:29:33 | mriedem | sahid: after going over the re-proposed spec yeah | |
| 14:29:37 | mriedem | i can do that shortly | |
| 14:32:50 | sahid | mriedem: cool, thank you | |
| 14:34:07 | sdague | mriedem: honestly, I expect just a lot of people approved stuff, and our normal races are all in there | |
| 14:34:19 | mriedem | sdague: the job isn't running | |
| 14:34:23 | mriedem | it's something with zuul | |
| 14:34:31 | sdague | mriedem: on what job? | |
| 14:34:39 | mriedem | http://status.openstack.org/zuul/ | |
| 14:34:45 | mriedem | gate-grenade-dsvm-neutron-multinode-ubuntu-trusty: queued | |
| 14:34:53 | mriedem | top job in the queue has been queued for nearly 19 hours | |
| 14:35:12 | AJaeger | sdague: I just pushed some notes on #openstack-infra about my findings | |
| 14:35:22 | mriedem | http://graphite.openstack.org/render/?from=-24hours&height=400&until=now&width=800&bgcolor=ffffff&fgcolor=000000&target=alias%28smartSummarize%28sumSeries%28stats_counts.zuul.pipeline.*.all_jobs%29,%271h%27%29,%27All%20Jobs%27%29&title=Zuul%20Jobs%20Launched%20%28per%20Hour%29&_t=0.5040648836791604#1488551678795 | |
| 14:35:43 | openstackgerrit | Balazs Gibizer proposed openstack/nova master: handle uninited fields in notification payload https://review.openstack.org/415857 | |
| 14:35:54 | sdague | oh | |
| 14:35:57 | sdague | neat | |
| 14:38:05 | mdbooth | Using only the cli, how do I determine which security group a particular instance is using? | |
| 14:38:30 | mdbooth | server show foo tells me name='default', but there are 4 of those | |
| 14:40:18 | mriedem | there is a secgroup-list command | |
| 14:40:21 | mriedem | or something like that | |
| 14:40:27 | mriedem | although we're removing that from novaclient | |
| 14:40:58 | mdbooth | Ah, I think I can get it from the port | |
| 14:41:01 | superdan | mriedem: hah, your create cell early thing is the same reason my fleetify patch is failing | |