Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2017-12-19
21:41:59 mriedem oolala
21:42:00 ludovic near future
21:42:21 mriedem ok. would be cool if you could test this out on a pike deployment.
21:42:32 mriedem to make sure the filter scheduler + placement is correctly handling this for you
21:42:42 ludovic That's why i asked if reservation and /or prioritizing exist under OpenStack
21:42:56 mriedem remember to remove the RamFilter in pike if you're using the FilterScheduler since (1) it's redundant and (2) it will remove the memory_mb claim in the compute
21:43:39 ludovic ok
21:44:40 ludovic don't you think it will be interesting to add the possibility to have spare Compute Nodes ?
21:45:02 ludovic with aggregate host spare for example
21:45:10 mriedem you mean build something into nova to mark specific computes as only used for evacuate?
21:45:37 ludovic or in a aggregat to propose the prioritinzing of important workload when evacuate
21:45:46 ludovic yes for example
21:46:35 mriedem yeah idk, maybe. i wouldn't want to change the evacuate api to pass through scheduler hints probably.
21:46:36 ludovic That will be ensure a very good SLA because the evacuate processus would be securized
21:46:54 mriedem i don't know how many deployments just have compute nodes lying around as spares for evacuate
21:47:20 mriedem you can also control which host is used client-side
21:47:21 mriedem as noted
21:48:00 mriedem so as a client, if you have a special "evacuate" host aggregate, you could round robin through those hosts and send it with the evacuate request
21:48:38 mriedem but, as your evacuate aggregate starts to fill it, it is no longer spare capacity
21:48:39 mriedem so...
21:48:46 mriedem *fill up
21:49:55 openstackgerrit Ed Leafe proposed openstack/nova master: Make conductor pass and use host_lists https://review.openstack.org/511358
21:49:56 openstackgerrit Ed Leafe proposed openstack/nova master: Change compute RPC to use alternates for resize https://review.openstack.org/526436
21:50:01 edleafe ^^ fixed pep8 booboo
21:50:33 ludovic it seems to not be easy to design ...
21:50:49 ludovic evacuate is temporary
21:51:14 ludovic until the source node repaired
21:51:47 ludovic so the goal is to failback and so free the evacuate aggregat
21:52:45 mriedem ludovic: sure, that's why it's not something built natively into nova
21:52:58 mriedem nova provides the API so a higher level service can orchestrate whatever you need here
21:53:57 mriedem edleafe: ack, will run the ironic patch on that
21:55:50 jose-phillips hi any idea
21:56:00 jose-phillips why im im getting this error on devstack
21:59:05 jose-phillips using qemu
21:59:05 jose-phillips can't apply process capabilities -1
22:00:51 rybridges for the record mriedem, i tried recompiling qemu like you said... took the latest version i could find here http://ftp.redhat.com/redhat/linux/enterprise/7Server/en/RHEV/SRPMS/
22:00:52 wind Hi, I'm trying to make a rest-api call from ironic to nova, was hoping if someone could gimme a hint how to do so ... Ironic.conf doesn't have anything in there for nova, so should i build a keystoneclient session, and then use that to retrieve the token, and compute-api endpoint and then send the GET & PUT Commands
22:00:56 rybridges still get the exact same erroro
22:01:21 wind Any suggestions would be really helpful.... I'm new to openstack
22:05:47 mnaser DEBUG?
22:05:47 mnaser 2017-12-19 22:04:53.593 44047 DEBUG nova.compute.resource_tracker [req-2858b7b9-d273-4347-aff3-dfa3fa10f4cd - - - - -] We're on a Pike compute host in a deployment with Ocata compute hosts. Auto-correcting allocations to handle Ocata-style assumptions. _update_usage_from_instance /usr/lib/python2.7/site-packages/nova/compute/resource_tracker.py:1042 <=== wouldnt this be nice if this was a warning instead of
22:06:17 mnaser given that I just found out Nova thinks there are Ocata hosts in my all-pike installation (i guess something is wrong somewhere)
22:08:02 ludovic mriedem: Just to be sure, in order to influence the host-evacuate process, i don't have other options than using host aggregate with flavor and extra_specs ?
22:08:36 mnaser ludovic: yes, but afaik migrations use the flavor and extra_specs that it had at the moment of provisioning
22:09:12 mnaser ex: if your flavor A had aggregate "foo" when a VM is booted, and you changed the aggregate to "bar", and then do a migration/evacuate, it will still try to look for "foo"
22:09:21 mnaser afaik that has been my experience but i might be wrong?
22:10:54 ludovic oh yes so it's necessary to provide two aggregates at the moemet of provisioning "prod" and "spare"
22:11:03 ludovic moment
22:11:15 mnaser well ideally if you launched an instance in 'prod', you dont want it to be evacuated into 'staging'
22:11:26 mnaser because maybe your host aggregate hardware in staging gets turned off at 5pm
22:13:30 mnaser oh would you look at that
22:13:54 mnaser "Unable to submit allocation for instance 42e2e0cd-0dd2-48c0-b873-ed9cd08a451a" .. placement returning 400, JSON does not validate: None is not of type 'string' ... instance['project_id'] == None somehow?!
22:14:12 mnaser does this ring any bells to anyone or should i start diving in
22:14:47 penick Working hard to get live migrations functional is like cranking on the handle for a jack-in-the-box. Except the thing in the box is a fist. And it punches you in the face.
22:15:21 ludovic mnaser: that sound JSON Filter not working no ?
22:16:03 mnaser penick: its always worked, but i guess this is a weird pike corner case, i see this - https://bugs.launchpad.net/nova/+bug/1701129 but it should be in pike which we're running
22:16:04 openstack Launchpad bug 1701129 in OpenStack Compute (nova) "Functional tests fail intermittently with 400 Bad Request from placement" [Low,Fix released] - Assigned to melanie witt (melwitt)
22:21:24 mnaser i guess for some reason RequestSpec is getting an empty project_id
22:21:47 openstackgerrit Lance Bragstad proposed openstack/nova master: Add scope_types to server policies https://review.openstack.org/525772
22:23:12 openstackgerrit Merged openstack/nova master: Convert ext filesystem resizes to privsep. https://review.openstack.org/517516
22:23:18 openstackgerrit Merged openstack/nova master: Move flushing block devices to privsep. https://review.openstack.org/519010
22:23:25 openstackgerrit Merged openstack/nova master: [placement] Separate API schemas (resource_class) https://review.openstack.org/520611
22:23:49 openstackgerrit Merged openstack/nova master: Update nova-status and docs for nova-compute requiring placement 1.14 https://review.openstack.org/526505
22:23:57 openstackgerrit Merged openstack/nova master: Deduplicate functional test code https://review.openstack.org/526227
22:24:04 openstackgerrit Merged openstack/nova master: Fix possible TypeError in VIF.fixed_ips https://review.openstack.org/527920
22:49:34 mnaser instance = common.get_instance(self.compute_api, context, id) <== would anyone know if this supplies project_id by default?
22:50:28 mnaser because that's the instance which is passed down to conductor and by the time its at the scheduler, instance.project_id == None which then in turn makes it fail the request to the placement api
22:55:12 mnaser further investigation - {"project_id": null, "user_id": "695d5f386eed440cb0e38455e1afdc9e", "allocations": [{"resource_provider": {"uuid": "5d5c5177-29bb-484f-9cc6-928360afa195"}, "resources": {"MEMORY_MB": 512, "VCPU": 2, "DISK_GB": 20}}, {"resource_provider": {"uuid": "4e43861e-ee36-40b7-ba7b-2239b46a1609"}, "resources": {"VCPU": 2, "MEMORY_MB": 512, "DISK_GB": 20}}]} .. for some reason, user_id comes in but
22:55:13 mnaser project_id doesn't. fwiw, this is a server created in 2015.
22:57:59 mnaser the user_id is the user of the one executing the live migration, not the owner of the instance oddly enough
23:06:45 mnaser ok.. request_spec record has project_id set to null for that vm
23:06:54 mnaser in the database
23:06:56 mnaser why and how.. :(
23:06:57 mriedem hmm, not sure why the project_id would be null
23:07:02 mriedem should come off the context
23:07:07 mriedem sorry, was on a call for the last hour
23:07:32 mnaser mriedem: no problem, its null because .. its null in the request_specs table too..
23:07:39 mnaser i wonder why
23:07:52 mriedem you said it's a really old instance right?
23:07:59 mnaser yes mriedem
23:08:25 mriedem ok reqspec is created here https://github.com/openstack/nova/blob/16.0.4/nova/compute/api.py#L899
23:08:30 mnaser the created_at for the requestspec is "2017-03-07 02:28:47"
23:08:34 mnaser but no updated_at
23:08:49 mriedem https://github.com/openstack/nova/blob/16.0.4/nova/objects/request_spec.py#L411
23:09:05 mriedem 2017-03-07 is ocata right?
23:09:25 mriedem i'm wondering if this was a request spec created for an older instance
23:09:29 mnaser it was
23:09:29 mriedem what's the created_at on the instance?
23:09:38 mnaser 2015 created_at, 2017 requestspec
23:09:59 mriedem https://github.com/openstack/nova/blob/stable/ocata/nova/objects/request_spec.py#L590
23:09:59 mriedem ok in ocata this is the routine for creating requestspecs for old instances
23:10:31 mriedem which https://github.com/openstack/nova/blob/stable/ocata/nova/objects/request_spec.py#L405
23:10:32 mriedem however,
23:10:42 mriedem if that's an admin context, from the online data migration, it won't have a project id...
23:11:03 mriedem https://github.com/openstack/nova/blob/stable/ocata/nova/cmd/manage.py#L776
23:11:05 mnaser which explains how we landed in this case
23:11:15 mriedem https://github.com/openstack/nova/blob/stable/ocata/nova/context.py#L313
23:11:17 mriedem yup
23:11:32 mnaser i guess its probably not the only one
23:11:51 mriedem probably not

Earlier   Later