Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-08-31
19:47:32 mriedem heh http://grafana.openstack.org/d/QBHIN5Smk/tempest-failure-rate?orgId=1
19:48:27 mriedem these dashboards are all using old job names
19:49:55 dansmith okay i thought this was spiking in the gate based on the discussion
19:50:07 mriedem not it's definitely not our worst failure
19:50:19 dansmith ack
19:50:25 mriedem http://status.openstack.org/elastic-recheck/#1789484
19:50:27 openstackgerrit Dan Smith proposed openstack/nova master: Move conductor wait_until_ready() delay before manager init https://review.openstack.org/598353
19:50:31 mriedem it's not nothing either
20:05:47 mriedem dansmith: +2
20:13:31 mriedem fried_rice: more details https://bugs.launchpad.net/nova/+bug/1790204/comments/1 - i expect reschedules would make any solution for that bug even messier
20:13:31 openstack Launchpad bug 1790204 in OpenStack Compute (nova) "Allocations are "doubled up" on same host resize even though there is only 1 server on the host" [Medium,Triaged]
20:13:59 fried_rice mriedem: ack. I'm subscribed.
20:17:20 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Fix image-defined numa claims during evacuate https://review.openstack.org/599062
20:17:51 mriedem we've got an easy gate fix here https://review.openstack.org/598353
20:17:58 mriedem would be good to get that in given the gate borkages of late
20:19:12 fried_rice mriedem, dansmith: +A
20:20:28 mriedem thanks
20:22:07 openstackgerrit Matt Riedemann proposed openstack/nova stable/rocky: Move conductor wait_until_ready() delay before manager init https://review.openstack.org/599064
20:51:58 openstackgerrit Eric Fried proposed openstack/nova master: Other host allocs may appear in gafpt during evac https://review.openstack.org/597301
20:55:10 openstackgerrit Eric Fried proposed openstack/nova master: Other host allocs may appear in gafpt during evac https://review.openstack.org/597301
20:55:45 openstackgerrit sean mooney proposed openstack/nova master: add caching to _build_regex_range https://review.openstack.org/599071
21:10:50 mriedem that wily old shelve https://bugs.launchpad.net/nova/+bug/1790221
21:10:50 openstack Launchpad bug 1790221 in OpenStack Compute (nova) "shelved offloaded server still shows old AZ while shelved" [Low,Triaged]
21:12:06 cfriesen mriedem: why is that bad?
21:12:22 cfriesen the unshelve api doesn't say anything about selecting a new AZ
21:12:41 mriedem cfriesen: a shelved instance isn't on a host,
21:12:49 mriedem note that the server details don't show a host or node when it's shelved
21:13:00 cfriesen mriedem: sure, but the AZ was selected at boot time as part of the boot criteria
21:13:00 mriedem if it's not on a host, it's clearly not in an AZ
21:13:06 mriedem nope
21:13:13 mriedem i didn't create the server with an az
21:13:23 mriedem so when i unshelve it, it can go into any othre az
21:14:02 mriedem related https://review.openstack.org/#/c/567701/
21:15:10 cfriesen I had thought that once we pick an AZ at boot it might have implications on which cinder backends are available, so you couldn't leave it afterwards
21:15:23 mriedem oh fuck no
21:15:36 mriedem related https://review.openstack.org/#/c/469675/
21:15:37 cfriesen cool, learned something new
21:15:59 mriedem we only give a shit about volume az if cinder.cross_az_attach=False, which it's not by default
21:16:09 mriedem if it is, well, good luck friend
21:16:12 mriedem b/c nova sucks at handling that
21:16:39 mriedem https://review.openstack.org/#/c/467674/ will show you exactly why
21:17:02 mriedem if you remove that nova fix as a dependency, you can't create a server with a pre-existing volume at all
21:17:08 mriedem because 'nova' != None
21:17:11 mriedem KABLAMMO
21:18:40 mriedem granted, my patch for that cross_az_attach fix is fugly
22:20:11 openstackgerrit Matt Riedemann proposed openstack/nova master: Null out instance.availability_zone on shelve offload https://review.openstack.org/599087
#openstack-nova - 2018-09-01
00:12:07 openstackgerrit Merged openstack/nova master: Fix image-defined numa claims during evacuate https://review.openstack.org/588657
00:37:43 openstackgerrit Merged openstack/nova master: Move conductor wait_until_ready() delay before manager init https://review.openstack.org/598353
21:25:25 openstackgerrit Matt Riedemann proposed openstack/nova stable/queens: Move conductor wait_until_ready() delay before manager init https://review.openstack.org/599200
#openstack-nova - 2018-09-02
00:35:10 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: implement reshaper for vgpu https://review.openstack.org/599208
00:35:15 mriedem fried_rice: bauzas: welcome to reshape city, population: libvirt ^
14:35:43 openstackgerrit Matt Riedemann proposed openstack/nova master: libvirt: implement reshaper for vgpu https://review.openstack.org/599208
#openstack-nova - 2018-09-03
00:16:08 openstackgerrit Takashi NATSUME proposed openstack/nova master: Fix a failure to format config sample https://review.openstack.org/597986
02:57:35 openstackgerrit Brin Zhang proposed openstack/nova master: Need further updates, no need to review https://review.openstack.org/599276
03:33:16 openstackgerrit Chen proposed openstack/nova master: Fix SOFT_DELETE vm_state string https://review.openstack.org/598482
06:29:48 prometheanfire https://github.com/openstack/nova/commit/50658eee4fe26a55854642119beeff46c0d0108a seems to be causing problems
06:30:00 prometheanfire 2018-09-03 01:29:14.620 11567 ERROR nova.objects.cell_mapping [req-0b912c05-d2bf-4e68-bcdf-7953c87dcc88 - - - - -] Failed to parse [database]/connection to format cell mapping: ValueError: expected ':' after conversion specifier
06:30:10 prometheanfire tonyb: think I found another one ^
06:30:44 prometheanfire not sure what's wrong with my db
06:30:52 prometheanfire I'm guessing my password isn't being escaped
06:32:04 tonyb prometheanfire: can you run nova in a venv and import cell_mapping.py and check the parse results with your netloc?
06:32:23 prometheanfire maybe the root?
06:32:23 prometheanfire 2018-09-03 01:29:14.919 11567 ERROR oslo_db.sqlalchemy.exc_filters [req-0b912c05-d2bf-4e68-bcdf-7953c87dcc88 - - - - -] DBAPIError exception wrapped from (psycopg2.ProgrammingError) relation "projects" does not exist
06:32:36 prometheanfire don't have a venv atm
06:33:02 tonyb prometheanfire: venvs are cheap, but if you hav nova installed as a site package you can just use that
06:33:02 prometheanfire another thing to blame not testing on postgres?
06:33:04 prometheanfire ya
06:33:25 tonyb prometheanfire: No I doubt that a postgres thing
06:34:58 prometheanfire my password actually is fine for the one in question
06:35:04 prometheanfire so probably not that
06:37:37 tonyb prometheanfire: :( Well I guess double check you're pointing at the right DB
06:39:57 prometheanfire it worked before the upgrade
06:43:34 prometheanfire not seeing anything in upgrade nodes or deprication notes or https://docs.openstack.org/nova/rocky/user/upgrade.html denoting any change needed
06:43:36 tonyb prometheanfire: ... Okay but that doesn't mean it wasn't using a deprecated feature that has gone between queens and rocky
06:43:45 tonyb prometheanfire: Okay
06:57:59 prometheanfire problem seems to be with this
06:58:00 prometheanfire https://github.com/openstack/nova/commit/50658eee4fe26a55854642119beeff46c0d0108a#diff-61f32a08cd0554ad482db124dcdc546aR128
06:58:04 prometheanfire the url.format is failing
06:59:28 prometheanfire https://gist.githubusercontent.com/prometheanfire/c98816d477331bcc0a0272adb93cc5e2/raw/c746f1b1780f1202286b9902178ff4e613329f44/gistfile1.txt
07:25:16 prometheanfire tonyb: it IS my password
07:51:59 openstackgerrit Zhenyu Zheng proposed openstack/nova-specs master: Make scheduling weight more granular https://review.openstack.org/599308
08:02:27 openstackgerrit Zhenyu Zheng proposed openstack/nova-specs master: Make scheduling weight more granular https://review.openstack.org/599308
08:14:09 prometheanfire ok, the nova_api database has a non-escaped database_connection in the cell_mappings table
08:15:24 prometheanfire which fixed THAT error at least
08:38:47 bauzas good morning stackers
08:57:42 openstackgerrit Matthew Thode proposed openstack/nova stable/rocky: Don't use '_TransactionContextManager._async' https://review.openstack.org/597421
09:11:58 openstackgerrit Chen proposed openstack/nova master: Fix SOFT_DELETE vm_state string https://review.openstack.org/598482
09:23:57 prometheanfire this isn't helping, for some reason re-running the service isn't using the file I just changed, wtf
09:24:36 prometheanfire https://gist.githubusercontent.com/prometheanfire/76b31b592b1780a72b32c289ab1bfb06/raw/b58c80c3d22860397781c53915d7350d6dbe0bab/gistfile1.txt
09:25:00 prometheanfire but async_ isn't defined at all on that line anymore, I editited it to be async
09:25:17 prometheanfire anyone have any idea where that's cached?
09:28:01 prometheanfire I've removed any pycache files I could find
09:30:38 openstackgerrit Merged openstack/nova master: libvirt: guest: introduce blockStats instead of domain.blockStats https://review.openstack.org/526833
09:55:11 openstackgerrit Surya Seetharaman proposed openstack/nova master: Making instance listing skipping down cells configurable https://review.openstack.org/592428
10:00:15 openstackgerrit Sujeet Banerjee proposed openstack/nova master: WIP: Added a periodic task for network management. https://review.openstack.org/599338
10:15:14 prometheanfire well, it looks like https://review.openstack.org/597421 may not be the right fix
11:23:28 gibi o/
11:27:09 sean-k-mooney gibi: o/
11:30:10 openstackgerrit Merged openstack/nova master: hardware: fix memory check usage for small/large pages https://review.openstack.org/532168
12:20:40 amarao How to access object attributes in policy json? I'd like to forbid some operations based on flavor. Is someone tried that before?

Earlier   Later