Earlier  
Posted Nick Remark
#openstack-nova - 2017-11-14
18:30:29 cburgess OK shoot
18:30:32 mriedem cburgess: today you can list in-progress live migrations for a given server
18:30:33 mriedem https://github.com/openstack/nova/blob/16.0.0/nova/db/sqlalchemy/api.py#L4789
18:30:38 mriedem if it's in one of those 4 states ^
18:30:50 mriedem however, you can only abort a running migration https://github.com/openstack/nova/blob/16.0.0/nova/compute/api.py#L3952
18:31:05 mriedem presumably because the migration transitions fast enough that it gets to the hypervisor and starts running
18:31:08 mriedem however,
18:31:21 mriedem we have this max_concurrent_live_migrations config option in compute which defaults to 1,
18:31:36 mriedem so you could be trying to move 10 instances off a host via live migration, and by default those are all going to queue up,
18:31:48 mriedem let's say you wanted to abort half of those (1 in 10 are running)
18:31:53 mriedem the abort would fail
18:32:47 cburgess mriedem Give me a few... sorry had someone talking to me
18:32:51 mriedem one solution is just bump up your max_concurrent_live_migrations value to 10, but then you hit issues with running a bunch of migrations at once
18:32:58 mriedem np, just looking for some ops feedback
18:33:55 alaski mriedem: there was http://git.openstack.org/cgit/openstack/nova-specs/tree/specs/backlog/approved/instance-tasks.rst
18:34:35 mriedem alaski: yup i found it later via google, i was mistakenly searching the nova docs
18:34:43 alaski cool
18:38:42 openstackgerrit Hongbin Lu proposed openstack/nova stable/pike: Handle exception on adding secgroup https://review.openstack.org/515418
18:40:33 openstackgerrit Hongbin Lu proposed openstack/nova stable/ocata: Handle exception on adding secgroup https://review.openstack.org/515421
18:42:44 jaypipes crlb: https://pastebin.com/maKwN7pM -- failures are reproduceable for me. Only suggestion I have is do an rm -rf .tox/ and then retry the tox -epy27,pep8. Other than that... perhaps you forgot to git review and push your latest changes to gerrit?
18:43:13 efried crlb If you run tox with -r it'll rebuild the venv.
18:43:23 openstackgerrit Merged openstack/nova stable/newton: Functional regression test for evacuate with a target https://review.openstack.org/516118
18:43:25 efried easier than rm -rf .tox
18:45:26 openstackgerrit Hongbin Lu proposed openstack/nova stable/pike: Handle exception on adding secgroup https://review.openstack.org/515418
18:46:13 openstackgerrit Hongbin Lu proposed openstack/nova stable/pike: Handle exception on adding secgroup https://review.openstack.org/515418
18:48:33 openstackgerrit Hongbin Lu proposed openstack/nova stable/ocata: Handle exception on adding secgroup https://review.openstack.org/515421
19:12:22 openstackgerrit Hongbin Lu proposed openstack/nova master: Handle not found error on taking snapshot https://review.openstack.org/511074
19:17:40 openstackgerrit Ken'ichi Ohmichi proposed openstack/nova master: Add error message on metadata API https://review.openstack.org/517152
19:20:59 crlb jaypipes: checked pastebin. The test_versions.py that you are running is not the latest, but patch set 2 updated it and dealt with the pep8 errors. If i try it review, I get: git review
19:20:59 crlb Creating a git remote called "gerrit" that maps to:
19:20:59 crlb ssh://crlb@review.openstack.org:29418/openstack/nova.git
19:20:59 crlb No changes between HEAD and gerrit/master. Submitting for review would
19:20:59 crlb be pointless.
19:22:23 mnaser crlb: did you commit anything on your local branch?
19:22:58 mnaser also, puppet fun, we suspect that we're seeing some weird timeout in puppet openstack intergration due to this change - https://github.com/openstack/puppet-nova/commit/df638e2526d2d957318519dfcfb9098cb7726095
19:23:23 mnaser we go from multiple threads in a single process, to single thread in multiple processes
19:23:55 mnaser but it seems to have spawned some strange timeouts with floating ip assignment. what's the nova team stance on this in terms of "best practice" ?
19:25:24 efried jaypipes Can I do stuff to this series? https://review.openstack.org/#/c/516778/
19:27:25 mnaser looks like devstack uses API_WORKERS for # of processes in uwsgi, and has an 'enable-threads' feature
19:36:13 crlb jaypipes. I'm sorry, I didn't realize that subsequent git reviews would give you different change numbers. The 2nd and 3rd patch sets are on change #518119.
19:45:13 mriedem dansmith: i reckon i should put out a release note for that cve on the stable branches, so i guess i'll do that right quick
19:45:17 mriedem and we can ram it through
19:46:14 efried crlb To be sure you are modifying the same review, the Change-Id in the commit message needs to remain the same. If you run `git commit` with the `--amend` flag, it'll make sure you're staying on the same change set. Otherwise (without `--amend`) it'll create a new change on top of the previous one and you'll wind up with separate reviews.
19:48:15 efried crlb At this point I believe you'll want to squash these changes together, assuming they're going after the same thing.
20:12:06 openstackgerrit Matt Riedemann proposed openstack/nova master: Add security release note for OSSA-2017-005 https://review.openstack.org/519738
20:12:15 mriedem sdague: dansmith: ^ if this looks ok, let's get it in and i'll do backports
20:21:31 dansmith mriedem: I dunno if that's common for a cve or not (to mention the cve but not describe it)
20:21:50 mriedem that's what the link is for
20:21:55 dansmith but if it is, it's cool with me
20:22:16 dansmith yeah, just seems sparse compared to what we'd normally do
20:22:23 mriedem let me find an old one
20:22:41 mriedem https://docs.openstack.org/releasenotes/nova/newton.html#security-issues
20:22:44 dansmith normally -> for a normal bug
20:23:06 mriedem https://docs.openstack.org/releasenotes/nova/newton.html#id15
20:23:07 dansmith mriedem: even that example is more descriptive
20:23:18 mriedem ok i'll wordsmith this a bit
20:24:07 dansmith I don't care that much, I would just expect it to say "this includes a fix for cve 123, which could result in instances rebuilt with images not normally allowed on their host" or something
20:26:40 openstackgerrit Merged openstack/nova stable/pike: Correct log message when removing a security group https://review.openstack.org/519583
20:27:01 openstackgerrit Merged openstack/nova master: Validate new image via scheduler during rebuild https://review.openstack.org/519662
20:27:08 openstackgerrit Merged openstack/nova stable/pike: Validate new image via scheduler during rebuild https://review.openstack.org/519672
20:27:15 openstackgerrit Merged openstack/nova master: Add FlavorPayload.description for versioned notifications https://review.openstack.org/514782
20:27:20 openstackgerrit Merged openstack/nova stable/pike: Pick ironic nodes without VCPU set https://review.openstack.org/513678
20:27:33 openstackgerrit Merged openstack/nova stable/ocata: Set group_members when converting to legacy request spec https://review.openstack.org/517860
20:27:50 openstackgerrit Merged openstack/nova stable/ocata: Validate new image via scheduler during rebuild https://review.openstack.org/519681
20:32:44 openstackgerrit Matt Riedemann proposed openstack/nova master: Add security release note for OSSA-2017-005 https://review.openstack.org/519738
20:32:45 mriedem dansmith: ^ added more detail
20:33:22 dansmith mriedem: thanks
20:34:24 mriedem jaypipes: melwitt: can you hit https://review.openstack.org/#/c/519738/ so i can start backports?
20:34:47 dansmith mriedem: melwitt is out for a couple weeks, AFAIK
20:34:52 mriedem orly
20:34:59 mriedem plural?
20:35:07 dansmith this week and part of next week I think
20:35:10 mriedem ok
20:35:25 dansmith through Nov 20
20:35:33 mriedem i'll give my jay and sean pings 5 more minutes and then i say we just approve
20:35:56 dansmith I can just fast approve. it's a reno.
20:35:58 mriedem ok
20:36:20 mriedem the fixes for newton are failing devstack, presumably because neutron is eol and things are just not going to work there anymore
20:37:13 mriedem if only we had LTS support...
20:37:34 dansmith um...
20:37:48 mriedem yeah, fudge, devstack in newton can't find stable/newton the branch, so it pip installs from master
20:37:55 mriedem farq
20:38:25 dansmith for nova?
20:38:33 mriedem no, devstack fails to setup
20:38:38 mriedem http://logs.openstack.org/19/516119/1/gate/legacy-tempest-dsvm-neutron-full/f24b24b/logs/devstacklog.txt.gz#_2017-11-14_17_24_29_768
20:38:40 mriedem b/c newton is eol
20:39:06 dansmith that's because neutron's branch is gone right?
20:39:15 dansmith any one project deletes their branch and devstack stops working yes?
20:39:46 mriedem yes
20:39:52 mriedem well, anything we need for our base install
20:40:03 dansmith so that's what I meant by "for nova?" ... meaning "for which project?"
20:40:13 mriedem i could remove neutron from ENABLED_SERVICES but then nova just won't work, unless nova-net
20:40:25 mriedem which in newton works w/o cellsv1....
20:40:26 mriedem :)
20:40:32 mriedem i think
20:40:44 dansmith or ask for the branch to be re-created based on the discussion at summit
20:40:54 dansmith mordred said it was easy to just resurrect that
20:41:07 mriedem it is, but then it would be EOL'ed next week ago
20:41:09 mriedem *again
20:41:39 dansmith or we start disabling jobs that don't work, per the discussion :)

Earlier   Later