Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-05-18
16:39:14 mdbooth dansmith zzzeek: Heh, yeah, its unlikely to be of interest to us :) It's of interest when you need to display text and, for eg, select characters with a mouse. I read a fantastic article about it recently but I can't find it right now.
16:39:15 sean-k-mooney sfinucan: looking at https://tools.ietf.org/html/rfc3629 section 10 "Security Considerations" we are both right.... the utf-8 encounding would support up to 6 bytes but such encounding produce illegal and only up to 4 bytes are vailid
16:40:05 mdbooth Questions like: what should pressing backspace do in a text editor are actually really hard to answer.
16:47:22 sean-k-mooney sdague: zzzeek mdbooth if ye just define the field as an NVARCHAR in sqlalchemy and set the coalation type for the table to utf8_bin or utf8_general_ci/utf8_unicode_ci the db should handel it its self
16:50:47 openstackgerrit Jianghua Wang proposed openstack/nova master: Skip policy rules on attach_network for none network allocation https://review.openstack.org/465903
16:55:31 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/nova master: Read Neutron port 'binding_profile' during boot https://review.openstack.org/449257
16:55:31 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/nova master: Add Neutron port capabilities to devspec in request https://review.openstack.org/451777
16:55:32 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/nova master: Format NIC features using os-traits definitions https://review.openstack.org/466051
16:57:27 openstackgerrit Rodolfo Alonso Hernandez proposed openstack/nova master: Read Neutron port 'binding_profile' during boot https://review.openstack.org/449257
18:17:20 openstackgerrit melanie witt proposed openstack/nova master: Cache database and message queue connection objects https://review.openstack.org/465042
18:18:27 ZZelle_ sfinucan, hi
18:34:09 marst edleafe: are you around?
18:36:45 openstackgerrit christopher.uhler proposed openstack/nova master: Libvirt volume driver for Veritas HyperScale https://review.openstack.org/443951
18:37:09 openstackgerrit Dan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack https://review.openstack.org/458537
18:53:44 cdent marst: edleafe is probably on a plane on his way to pycon
18:59:08 marst cdent: thanks! I've just read his blog post
19:15:18 openstackgerrit Sean Dague proposed openstack/nova master: see if we can use pg wheels with no bindep https://review.openstack.org/466081
19:26:09 mriedem dansmith: did you figure out http://logs.openstack.org/37/458537/20/check/gate-tempest-dsvm-multinode-live-migration-ubuntu-xenial/ab364da/logs/testr_results.html.gz ?
19:47:47 openstackgerrit Cedric Brandily proposed openstack/nova master: Correct _ensure_console_log_for_instance implementation https://review.openstack.org/466088
19:48:56 mriedem this seems wrong http://logs.openstack.org/42/465042/6/check/gate-nova-tox-functional-py35-ubuntu-xenial/9829a45/console.html#_2017-05-18_18_43_03_020297
19:49:00 mriedem b"KeyError: 'allocations'"
19:49:56 dansmith mriedem: I found the hook yeah, still working on a fix
19:50:07 mriedem ok,
19:50:21 mriedem i'm working on disabling the grenade live migration job from running on pike until we fix it, or queens
19:50:28 mriedem since right now it's just burning nodes
19:51:53 mriedem this is very weird - http://logs.openstack.org/42/465042/6/check/gate-nova-tox-functional-py35-ubuntu-xenial/9829a45/console.html#_2017-05-18_18_43_03_020297 it's like in the middle of a functional test, we got a different response from placement
19:52:07 mriedem like it's a GET / response
19:52:21 mriedem but was expecting GET /resource_providers/d41180aa-047e-4bc3-b0f3-11532683400d/allocations
19:54:44 openstackgerrit Cedric Brandily proposed openstack/nova master: Correct _ensure_console_log_for_instance implementation https://review.openstack.org/466088
19:58:31 melwitt mriedem: yeah, I didn't know what to make of that
20:03:21 mriedem could have something to do with running those services under eventlet in our functional tests?
20:20:29 efried Where does the hashy thing at the end of a release note .yaml's basename come from?
20:21:36 openstackgerrit Dan Smith proposed openstack/nova master: DNM: Canary test against multi-cell devstack https://review.openstack.org/458537
20:21:54 artom efried, the 'reno' command does it
20:23:04 efried artom Is there a doc somewhere that describes the release note process?
20:23:12 efried (Cause your answer just raised bigger questions ;-)
20:23:35 artom efried, not that I know of, but if you have a devstack env somewhere reno should be in there
20:23:35 openstackgerrit Matt Riedemann proposed openstack/nova master: Skip ceph in grenade live migration job due to restart failures https://review.openstack.org/466097
20:23:36 mriedem sdague: i think i've found a poor man's compromise ^
20:23:44 artom efried, in which case reno --help will help :)
20:24:01 mriedem efried: artom: https://docs.openstack.org/developer/nova/releasenotes.html
20:24:06 efried artom Yeah, found reno --help; it kinda assumes you know what youre doing already.
20:24:10 efried Okay, thank you artom
20:24:16 mriedem the hash is auto-generated
20:24:26 mriedem tox -e venv -- reno new <your reno name>
20:25:01 mriedem dhellmann said he's found that sometimes people copy renos and re-use the same hash which messes things up
20:25:26 efried I guessed that would be a no-no
20:25:33 efried Thanks mriedem & artom
20:25:42 mriedem yeah, plus reviewers wouldn't be able to eyeball that
20:25:58 mriedem seems the reno docs build itself should fail if that happens
21:25:13 openstackgerrit Alex Szarka proposed openstack/nova master: Add method for verify multiple notifications https://review.openstack.org/465526
21:25:33 openstackgerrit Alex Szarka proposed openstack/nova master: [WIP] Transform instance.exists notification https://review.openstack.org/403660
21:33:10 openstackgerrit Monty Taylor proposed openstack/nova master: WIP Show results of nested parameters in api-ref https://review.openstack.org/464277
21:33:10 openstackgerrit Monty Taylor proposed openstack/nova master: WIP Add description of server addresses objects https://review.openstack.org/464331
21:39:32 cfriesen mriedem: you around?
21:40:03 mriedem sort of
21:40:50 cfriesen mriedem: got a weird scenario...upgrades are in progress, one compute node hasn't been upgraded. It gets rebooted without being upgraded and hits exception.ServiceTooOld
21:41:16 cfriesen mriedem: are we just expecting people to upgrade in that case?
21:41:35 openstackgerrit Alex Szarka proposed openstack/nova master: Add method for verify multiple notifications https://review.openstack.org/465526
21:42:34 cfriesen mriedem: actually, thinking about it a bit I'm not clear why the version of that compute node isn't being seen in the DB
21:42:44 openstackgerrit Alex Szarka proposed openstack/nova master: [WIP] Transform instance.exists notification https://review.openstack.org/403660
21:44:01 cfriesen mriedem: maybe the service is still being forced down by our management software, causing its version to be ignored by service_get_minimum_version()
21:45:53 mriedem that's a dansmith question
21:46:50 mriedem cfriesen: it's blowing up here right? https://github.com/openstack/nova/blob/master/nova/objects/service.py#L307
21:47:14 cfriesen mriedem: yep.
21:47:39 cfriesen down in the sql code we ignore the version of "forced-down" services, so maybe that's it
21:49:12 mriedem right this query https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api.py#L490
21:50:46 mriedem so all of your other computes are newton, and you've got a mitaka forced-down compute, and you're restarting it and it's blowing up because that mitaka compute is older than the minimum for all of the other computes, which is newton
21:51:38 mriedem so upgrade that compute to newton
21:51:57 cfriesen mriedem: yeah, I'm not certain it's forced down but it's the only thing that makes sense
21:52:06 mriedem do you have the service record from the db?
21:53:06 cfriesen mriedem: could probably track it down. would indeed be able to tell from that. I'm just reviewing a proposed change, wasn't actually my issue. one of the guys wants to bypass the check and I'm not a big fan.
21:53:24 mriedem i mean you could PUT /os-services/force-down and mark that service as forced_down=False, which would change the min service version check to say mitaka is the oldest, but that might give you other issues if other things are running thinking that newton is the minimum
21:54:34 cfriesen we're not using auto-rpc versioning (yet, at least) so during an uprade I think it'd be okay as long as we haven't set the rpc version to newton yet. will have to go get more info I think.
21:56:16 openstackgerrit Matt Riedemann proposed openstack/nova master: Skip ceph in grenade live migration job due to restart failures https://review.openstack.org/466097
22:00:08 mriedem cfriesen: did you happen to try this out in your live migration recreate with the 16 ports and network-changed event issues? https://review.openstack.org/#/c/465787/
22:00:33 mriedem before i go too far down that rabbit hole (probably next week at this rate) it'd be good to know if it helps at all
22:08:00 cfriesen mriedem: related to the other issue, it turns out that if you force the "old" nova-compute down, then try to remove the "force-down" it will fail because Service.save() checks the minimum version
22:08:28 cfriesen mriedem: haven't tried that patch yet, but I plan to.
22:08:29 mriedem oh that's a fun chicken and egg
22:08:45 cfriesen isn't it just. :)
22:14:54 mriedem well, seems like a bug to report. we could bypass the service min version check on save() i'd think, but i'm not sure what kind of weird situations that puts us in
22:15:35 mriedem again, like if you've already restarted control services that are caching what they think is the min service version (newton) and all of a sudden you have a running mitaka compute service
22:15:36 cfriesen mriedem: just for fun, the original scenario was that we were trying to take down the compute node to upgrade it, so we marked nova-compute as forced-down, then we tried to down the neutron agent and hit an exception, so we tried to roll back and enable nova-compute and hit exception.ServiceTooOld
22:16:11 mriedem cfriesen: why not just disable the service?
22:16:11 cfriesen not sure there's a good solution here.
22:16:33 mriedem PUT /os-services/disable-log-reason
22:16:41 mriedem that should take that nova-compute out of scheduling
22:17:36 cfriesen mriedem: that's a good point...usually we only force down on a fault. will follow up.
22:21:18 mriedem honestly i'm not sure what the use case is for forced-down which is different from disabling the service
22:21:49 mriedem i think it has to do with external tooling forcing the service to be 'down' rather than waiting for the service group api check to do it
22:22:01 mriedem we could document that better in the api ref
22:22:21 mriedem like, what does doing this actually mean, what impacts / side effects does it have, etc
22:22:31 mriedem cfriesen: ^ probably another bug to open for lack of clear docs on these
22:22:47 mriedem https://specs.openstack.org/openstack/nova-specs/specs/liberty/implemented/mark-host-down.html was the spec
22:23:34 mriedem oh i see, the idea is external service finds out the host is about to die, forces it down and then evacuates the instances off that host
22:23:41 mriedem since evacuate checks this flag
22:24:11 mriedem we should state in the api-ref that forced-down is not something for routine upgrade/maintenance
22:24:44 mriedem as documented here https://docs.openstack.org/ops-guide/ops-maintenance-compute.html#planned-maintenance
22:30:02 openstack Launchpad bug 1691871 in OpenStack Compute (nova) "forced-down vs service disable is not documented well in the compute API reference" [Medium,Confirmed]

Earlier   Later