Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2019-08-20
19:11:25 mnaser yep
19:11:51 openstackgerrit Merged openstack/python-novaclient master: Microversion 2.75 - Multiple API cleanup changes https://review.opendev.org/676275
19:28:53 openstackgerrit Merged openstack/python-novaclient master: API microversion 2.76: Add 'power-update' external event https://review.opendev.org/666792
19:58:36 mriedem efried: i guess t-2 should be closed out in lp huh
19:58:39 mriedem as of july 25
20:00:16 mriedem i closed out the bps for 2.75 and 2.76
20:07:49 efried mriedem: I don't know what you're saying.
20:08:04 mriedem efried: the t-2 milestone in launchpad for nova is still open
20:08:13 mriedem even though we're past the t-2 (fake-ish) milestone
20:08:24 mriedem https://wiki.openstack.org/wiki/Nova/Train_Release_Schedule
20:08:36 mriedem melwitt: https://review.opendev.org/#/c/672840/8 is busted - you remove the call to archive
20:08:38 mriedem *removed
20:08:56 mriedem i think my comment in PS4 caused the confusion, i just meant to remove the $conf passed to the bash function since we didn't need it anymore
20:09:37 efried mriedem: I didn't know launchpad tracked "milestones". How do I look at that?
20:09:57 efried (and, not to put too fine a point on it, but does anyone care?)
20:10:14 mriedem https://launchpad.net/nova/+series
20:10:55 mriedem efried: not really since the release team stopped doing milestone releases,
20:11:01 mriedem we really only have them for internal tracking
20:11:05 efried so... I can go into it and uncheck Active and that's it?
20:11:11 mriedem i.e. t-2, nova-spec freeze
20:11:41 mriedem in the past i've created a release
20:11:46 mriedem which closes it out
20:11:54 melwitt mriedem: argh, sorry. I saw the commands were in a function and then I promptly didn't realize that was the func call. will fix
20:12:15 mriedem efried: i'm not sure if marking it as inactive just removes it from targeting for bugs and blueprints but still shows up in history
20:12:28 mriedem since we don't create an actual release (we don't tag the repo), marking as not active is probably more correct
20:13:06 efried made it disappear from the little flow charty thingy.
20:17:35 mriedem i still see it here https://launchpad.net/nova/train
20:17:42 mriedem but not marked active https://launchpad.net/nova/+milestone/train-2
20:17:44 mriedem which means,
20:18:05 mriedem it doesn't show up as a milestone target either, which is what we want
20:18:09 mriedem so we're good
20:23:03 efried \o/
20:38:26 openstackgerrit Eric Fried proposed openstack/nova master: objects: Rename 'nova.objects.instance_numa_topology' https://review.opendev.org/671789
20:38:26 openstackgerrit Eric Fried proposed openstack/nova master: libvirt: Remove unnecessary try-catch around 'getCPUMap' https://review.opendev.org/671790
20:38:27 openstackgerrit Eric Fried proposed openstack/nova master: libvirt: Remove unnecessary argument https://review.opendev.org/674396
20:38:28 openstackgerrit Eric Fried proposed openstack/nova master: objects: Remove legacy '_from_dict' functions https://review.opendev.org/537414
20:38:28 openstackgerrit Eric Fried proposed openstack/nova master: claims: Remove useless caching https://review.opendev.org/671791
20:38:29 openstackgerrit Eric Fried proposed openstack/nova master: trivial: Rename exception argument https://review.opendev.org/671795
20:38:29 openstackgerrit Eric Fried proposed openstack/nova master: Remove 'hardware.get_host_numa_usage_from_instance' https://review.opendev.org/671797
20:38:30 openstackgerrit Eric Fried proposed openstack/nova master: Remove 'hardware.host_topology_and_format_from_host' https://review.opendev.org/671798
20:38:30 openstackgerrit Eric Fried proposed openstack/nova master: Remove 'hardware.instance_topology_from_instance' https://review.opendev.org/671799
20:38:31 openstackgerrit Eric Fried proposed openstack/nova master: Rework 'hardware.numa_usage_from_instances' https://review.opendev.org/672565
20:38:31 openstackgerrit Eric Fried proposed openstack/nova master: tests: Split NUMA object tests https://review.opendev.org/672336
20:38:32 openstackgerrit Eric Fried proposed openstack/nova master: Add '[compute] cpu_dedicated_set' option https://review.opendev.org/671792
20:38:32 openstackgerrit Eric Fried proposed openstack/nova master: libvirt: Start reporting PCPU inventory to placement https://review.opendev.org/671793
20:38:32 openstackgerrit Eric Fried proposed openstack/nova master: libvirt: '_get_(v|p)cpu_total' to '_get_(v|p)cpu_available' https://review.opendev.org/672693
20:38:33 openstackgerrit Eric Fried proposed openstack/nova master: trivial: Rewrap definitions of 'NUMACell' https://review.opendev.org/674395
20:38:34 openstackgerrit Eric Fried proposed openstack/nova master: hardware: Differentiate between shared and dedicated CPUs https://review.opendev.org/671800
20:38:34 openstackgerrit Eric Fried proposed openstack/nova master: objects: Rename 'fields' import to 'obj_fields' https://review.opendev.org/674103
20:38:35 openstackgerrit Eric Fried proposed openstack/nova master: libvirt: Start reporting 'HW_CPU_HYPERTHREADING' trait https://review.opendev.org/675571
20:38:46 melwitt mriedem: the archive command should go after 'set -e' too right? or no?
20:39:29 melwitt oh, it shouldn't bc we expect it to return 1 several times
20:40:54 mriedem melwitt: i think it can because the functions will only return a non-0 if they fail
20:41:19 melwitt mriedem: oh, I see. the function, not the individual commands. ok thanks
20:41:21 mriedem that's one issue i've meant to cleanup in that script is if the archive or purge fails the script wouldn't actually fail and the job wouldnt' fail
20:42:29 melwitt yeah, I see
20:42:57 openstackgerrit Matt Riedemann proposed openstack/nova master: Deprecate Aggregate[Core|Ram|Disk]Filters https://review.opendev.org/673496
20:45:44 openstackgerrit Matt Riedemann proposed openstack/nova master: Deprecate Aggregate[Core|Ram|Disk]Filters https://review.opendev.org/673496
20:45:48 mriedem sean-k-mooney: stephenfin: i've addressed comments in ^ and am +2 now
20:48:34 openstackgerrit melanie witt proposed openstack/nova master: Verify archive_deleted_rows --all-cells in post test hook https://review.opendev.org/672840
20:55:39 mriedem efried: looking at the grenade failures in https://review.opendev.org/#/c/676684/, it appears to be segfaults killing q-svc
20:55:59 mriedem Aug 20 15:53:30 ubuntu-bionic-rax-dfw-0010301045 kernel: /usr/bin/python[21875]: segfault at a9 ip 000055aee57ec9e4 sp 00007ffed66329d8 error 4 in python2.7[55aee5716000+301000]
20:56:05 mriedem which i've seen locally but not so much in the gate
20:56:11 efried mriedem: segfault + switch-to-something-called-binary == suspicious
20:56:34 mriedem but these jobs aren't even using postgresql
20:56:50 efried I didn't go back and check the previous failures
20:57:02 efried but it seemed like *all* the grenade jobs failed *every* patch set.
20:57:08 efried (anecdotal ^ )
20:58:07 mriedem the pips are installed
20:58:08 mriedem psycopg2==2.8.3 psycopg2-binary==2.8.3
20:58:08 efried Anyway, I have a definite -1 on the requirements-check problem. If the grenade thing turns out to be unrelated, great.
20:58:26 mriedem the good news is there are hits in logstash
20:58:29 mriedem http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22Unexpected%20exception%20in%20API%20method%3A%20ConnectFailure%3A%20Unable%20to%20establish%20connection%20to%5C%22%20AND%20message%3A%5C%22v2.0%2Fports%3Fdevice_id%5C%22%20AND%20message%3A%5C%22('Connection%20aborted.'%2C%20BadStatusLine('No%20status%20line%20received%20-%20the%20server%20has%20closed%20the%20connection'%2C))%5C%22%20AND%20tags
20:58:30 mriedem 5C%22screen-n-api.txt%5C%22&from=7d
20:58:35 mriedem the bad news is they are only on that one change
20:58:44 efried is that bad news?
20:58:56 efried It seems like it would be further evidence that the problem is *there*.
21:02:45 mriedem well it's not good news
21:03:19 mriedem i've seen this same thing in my devstack vms i've created lately b/c there is too much filling up a single 8gb ram/8vpu vm even after stripping the services to pretty bare bone
21:03:25 mriedem clarkb has posted about it in the ML so it's a known issue
21:03:36 mriedem but not many people working on it
21:05:21 efried is it possible that the -binary thingy is just sucking up too much memory?
21:07:15 my_nd hi there, is there a way to configure a compute node to use only some % of their available resources to host instances?
21:10:40 mriedem my_nd: yes, you can configured reserved cpu, ram and disk for the host
21:10:58 mriedem https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.reserved_host_cpus
21:11:05 mriedem https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.reserved_host_disk_mb
21:11:13 mriedem https://docs.openstack.org/nova/latest/configuration/config.html#DEFAULT.reserved_host_memory_mb
21:12:07 my_nd thanks mriedem
21:13:13 mriedem efried: postgresql shouldn't even be running
21:13:35 mriedem i don't see it in https://a6b55b727208d469fce9-a76e08006e67cd388b33580b35bd0960.ssl.cf2.rackcdn.com/676684/7/check/nova-grenade-live-migration/8626716/logs/screen-peakmem_tracker.txt.gz anyway
21:13:44 mriedem mysqld is in there
21:16:23 mriedem unfortunately peakmemtracker seems to die before we start seeing segfaults
21:16:40 mriedem top memory usage is unsurprisingly mysqld, neutron, and nova (api and compute)
21:18:01 efried I can't make that logstash query work, because it split lines on me. Can you paste just the text from the query box?
21:18:23 mriedem message:"Unexpected exception in API method: ConnectFailure: Unable to establish connection to" AND message:"v2.0/ports?device_id" AND message:"('Connection aborted.', BadStatusLine('No status line received - the server has closed the connection',))" AND tags:"screen-n-api.txt"
21:18:29 efried or you can just tell me: are there hits from the earlier patch sets?
21:18:55 mriedem 6 and 7 yes
21:19:18 efried mm
21:19:37 mriedem nothing really changed much since PS5 though...
21:19:46 efried those are all from nova-grenade-live-migration.

Earlier   Later