Earlier  
Later
New Search
Posted Nick Remark
#openstack-nova - 2018-07-30
20:49:07 dansmith and if you're going to, you should probably be using ceph or cinder backing
20:49:24 s10 cfriesen: this is not the issue, if recreate:
20:49:24 s10 migration = {'source_compute': instance.host,
20:49:24 s10 'dest_compute': self.host, }
20:49:24 s10 self.network_api.migrate_instance_finish(context,
20:49:24 s10 instance,
20:49:25 s10 migration)
20:49:39 s10 This sets ports for instances on new host
20:49:52 sean-k-mooney s10: also if your using routed netwroking you need to make sure your new host is on the same segment.
20:49:54 cfriesen__ I'm talking about allocations, resource tracking, etc
20:50:29 cfriesen__ placement, PCI device and CPU and hugepage claims, etc
20:50:42 s10 dansmith: yes, but what if we want as much IOPS for instances as possible - this could only be archived with local NVMes, so red etc is not and option
20:51:51 dansmith ack, fair enough
20:52:00 sean-k-mooney s10: you dont need to keep all of your data on the nvme drivres. having an nvme backed epheral disk for scratch space or chacing is a more cloud friendly approch
20:52:16 dansmith yup
20:52:35 dansmith there's this triangle..
20:53:04 dansmith http://katharine-writes.com/wp-content/uploads/2014/03/Quality-Triangle.jpg
20:53:54 openstackgerrit Merged openstack/nova master: Add description for placement 1.26 https://review.openstack.org/587067
20:56:13 sean-k-mooney dmcache and bcache in the kernel are actully great ways to cache /var/lib/nova/instances mounts from remote network filesystem using local nvmedrives
20:57:13 sean-k-mooney i rembere looking to see if qemu could be configured to cache its remote disks via local fast storage but never found a way
20:57:30 dansmith or just use something capable of doing it natively
20:57:52 openstackgerrit Matt Riedemann proposed openstack/nova master: Retry decorator fix for instances which go into ERROR state during bulk delete https://review.openstack.org/563418
20:57:53 mriedem jaypipes: melwitt: there you go ^ easy peasy
20:58:06 s10 reserved host is a complete replica of the failed host, same cpus, no pic devices, same memory, in same network segment, only thing that should be changed is the hostname...
20:58:26 sean-k-mooney ya i was trying to see if i could figure out how to cache vms backed by ceph using nvdim persistent memory without modifying qemu
20:59:18 dansmith yeah for transparent backing of block devices, dm-cache is the way
20:59:51 melwitt mriedem: thanks
21:00:20 sean-k-mooney s10: i was pritty sure we had a way to override the hostid of a compute node but i could not find it in the conf. that bacsically all you need. e.g. a way to forst the agent to think its on the old host without actully using the old host name on the phyical host
21:00:49 dansmith ...or just transport the damn system disk :)
21:00:57 dansmith you're walking across the aisle anyway :)
21:01:32 s10 sean-k-mooney: i'm not sure, that we can tell that hostid to the neutron-ovs-agent same way :(
21:01:40 sean-k-mooney dansmith: ya that would likely be quicker then any workaround we can comup with bar adding a compute node uuid to the compute conf
21:01:58 sean-k-mooney s10: the neutron agent definetly supports it
21:02:07 sean-k-mooney its nova that is missing it
21:02:32 melwitt yeah, I was gonna say, given all the things that would need to be updated, copying over the system disk sounds like the easiest and fastest way
21:03:43 sean-k-mooney dansmith: melwitt out of interest is there any reason we could not allow an operator to choos the compute node uuid by setting it in the conf?
21:04:30 dansmith sean-k-mooney: it's internal and more complicated to let them choose
21:04:43 dansmith right now it can't change unless we change it or do things
21:05:00 dansmith I can't really see a compelling reason to let them change it
21:06:48 sean-k-mooney im not saying it a good idea. its a great way to shoot yourself in the foot and mess up our resouce tracking. but it would make s10 usecase doable without a lot of db hacking
21:07:42 s10 Or can I just change compute_driver to Fake on reserved node, execute evacuate to this node and then just put disks from old host, and than change config back and restart nova-compute...
21:08:47 s10 compute_driver=fake.FakeDriver. and than back to libvirt.LibvirtDriver.
21:12:42 sean-k-mooney s10: that will likely take longer then pulling the os driver if your goal is to minimise guest downtime
21:17:54 s10 sean-k-mooney: nova host-evacuate --target-host --force can be executed immediately after host failure. Same time, while instances are being "evacuated" to the fake host, support will pull disks from failed server and put them in the reserved, right after that we stop nova-compute, mount disks from old host, change nova config, start it again, and finally nova reboot for all instances. This will take much less time than moving system m.2 disk
21:19:17 s10 Only physical operation that should be performed is moving disks. Other way requires opening the lid on the top of the server with dismounting it out of the rack.
21:19:36 mriedem i highly suggest not using the --force option
21:19:51 mriedem since it bypasses the scheduler and can mess things up if the target host is not actually a proper fit for the instance
21:20:05 dansmith also,
21:20:17 dansmith the fake driver will not have the right inventory in placement
21:20:31 dansmith so your scheduler or conductor-based allocations would be all fubar
21:20:33 sean-k-mooney mriedem: in this case s10 is forcing a migrate to a fake libvirt node so not sure it applies
21:20:41 dansmith it definitely applies
21:21:11 sean-k-mooney dansmith: oh because it skip the schduler and never update placement
21:21:12 mriedem i'm not sure why using the fake virt driver in any of this is a good idea
21:21:15 s10 yep. that is not the right way because of the inventory in placement. if we can't change it after.
21:21:47 dansmith sean-k-mooney: not just that, but the inventory in placement that will come from the fake driver won't look anything like the inventory that the system will provide after you change it to libvirt
21:21:53 openstackgerrit Merged openstack/nova master: Updated AggregateImagePropertiesIsolation filter illustration https://review.openstack.org/586317
21:22:08 sean-k-mooney s10: dumb question. why can you not put the os dirve in one of the hot swap bays so its just as easy to move as the other disks
21:22:29 dansmith sean-k-mooney: m2 drives on the motherboard are getting common for system drives today
21:22:35 dansmith but of course, he could do that :)
21:22:47 dansmith or buy systems with enough hot-swap bays
21:22:53 dansmith or tell his vendor he doesn't like their placement of the m2
21:23:01 dansmith or use a system with a rear-mounted hot-swap bay for system drives
21:23:03 dansmith lots of options
21:23:32 s10 sean-k-mooney: because we have 4 slots for disk in 1U server. we use it all for storage disks or have to use one of them for system.
21:25:41 sean-k-mooney s10: ok but i have racked a lot of sytems in the intel dev lab. it really does not take that much time to pull an internal drive. for an m2 it 1 screw most of the time and you often dont need to even unrack the server.
21:26:48 sean-k-mooney s10: if you really do have this requirement then fair enough but it seams like premature optimization unless you expect to have to do this often
21:28:13 s10 sean-k-mooney: you are right, this is a premature optimisation.
21:45:15 openstackgerrit Eric Fried proposed openstack/nova master: [placement] Add /reshaper handler for POST https://review.openstack.org/576927
21:45:16 openstackgerrit Eric Fried proposed openstack/nova master: reshaper: Look up provider if not in inventories https://review.openstack.org/585033
21:45:17 openstackgerrit Eric Fried proposed openstack/nova master: Make get_allocations_for_resource_provider sane https://review.openstack.org/584598
21:45:18 openstackgerrit Eric Fried proposed openstack/nova master: Report client: Real get_allocs_for_consumer https://review.openstack.org/584599
21:45:19 openstackgerrit Eric Fried proposed openstack/nova master: Report client: get_allocations_for_provider_tree https://review.openstack.org/584648
21:45:20 openstackgerrit Eric Fried proposed openstack/nova master: Report client: _reshape helper, placement min bump https://review.openstack.org/585034
21:45:21 openstackgerrit Eric Fried proposed openstack/nova master: Report client: update_from_provider_tree w/reshape https://review.openstack.org/585049
21:45:22 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Compute: Handle reshaped provider trees https://review.openstack.org/576236
22:41:27 openstackgerrit Eric Fried proposed openstack/nova master: WIP: Compute: Handle reshaped provider trees https://review.openstack.org/576236
23:07:33 mriedem cfriesen__: where is the code for the nova api proxy thing for starlingx?
23:07:37 mriedem is that in the nova staging repo?
23:10:48 mriedem i only see https://github.com/openstack/stx-nfv/tree/master/nova-api-proxy
23:14:40 openstackgerrit Artom Lifshitz proposed openstack/nova stable/pike: Refactor _build_device_metadata https://review.openstack.org/587248
23:14:41 openstackgerrit Artom Lifshitz proposed openstack/nova stable/pike: Consider hostdev devices when building metadata https://review.openstack.org/587249
23:27:50 cfriesen__ mriedem: one sec
23:32:43 cfriesen__ mriedem: You've got the right code. The proxy passes most things directly through to the "real" nova-api, while a subset of calls (see https://github.com/openstack/stx-nfv/blob/master/nova-api-proxy/nova-api-proxy/nova_api_proxy/apps/acceptor.py#L114) get forwarded to what we call the "VIM", which I think is now at "https://github.com/openstack/stx-nfv/tree/master/nfv/nfv-vim/nfv_vim"
23:33:48 cfriesen__ mriedem: the reason for this is so that the vim can remain up-to-date about the state of the instance even in the face of races
23:34:44 cfriesen__ I'm not very experienced in that area of the code...mostly I can just ignore it.
23:37:34 cfriesen__ mriedem: the vim handles things like automated recovery if a qemu process crashes, or a compute node goes down.
23:38:25 cfriesen__ mriedem: also things like pushing all instances off a node if we want to take it down for maintenance
#openstack-nova - 2018-07-31
01:03:59 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in test_compute_api.py (4) https://review.openstack.org/568462
01:04:26 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (5) https://review.openstack.org/570842
01:04:41 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (6) https://review.openstack.org/571330
01:05:03 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (7) https://review.openstack.org/571992
01:05:20 openstackgerrit Takashi NATSUME proposed openstack/nova master: Remove mox in libvirt/test_driver.py (8) https://review.openstack.org/571993
01:20:02 openstackgerrit Merged openstack/nova master: Retry decorator fix for instances which go into ERROR state during bulk delete https://review.openstack.org/563418
02:01:18 openstackgerrit Merged openstack/nova master: Set default of oslo.privsep.daemon logging to INFO level https://review.openstack.org/586643
02:06:07 alex_xu mriedem: really don't know whether we need Chinese in metadata, I thought the Chinses will be used in the UI, not sure using Chinese in the metadata is good idea. It really a production question, Kevin_Zheng is good answer that I guess.
02:08:12 Kevin_Zheng alex_xu: people are using api too, I think the patch is about metadata key?
02:09:23 alex_xu Kevin_Zheng: the bug reported for key and value
02:10:01 Kevin_Zheng I will check, I remember we already support
02:10:06 Kevin_Zheng In value

Earlier   Later