| Posted | Nick | Remark | |
|---|---|---|---|
| #openstack-sdks - 2018-05-01 | |||
| 12:19:39 | mordred | mnaser: seems like a good use case - I'm looking through the code to see if I can see where endpoint_override isn't getting passed ... | |
| 12:20:11 | mordred | mnaser: ah. there it is. it's in the part where it's not getting passed | |
| 12:22:17 | mnaser | mordred: currently doing this.. but it also feels a little dirty, i could be doing it the wrong way | |
| 12:22:26 | mnaser | i didnt find a way to just connect to a region with overrides | |
| 12:22:34 | mordred | yah - there'sa bug - patch coming | |
| 12:23:01 | mnaser | (the reason i'm overriding is because i want to test the *local* instance using that monitoring agent rather than going through load balancer, etc) | |
| 12:23:03 | mnaser | and cool thank you! | |
| 12:24:30 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Honor endpoint_override for get_session_client https://review.openstack.org/565489 | |
| 12:24:38 | mordred | mnaser: ^^ | |
| 12:24:55 | mnaser | yay | |
| 12:24:59 | mnaser | let me try and test that locally | |
| 12:29:27 | mnaser | mordred: your patch is missing a , after 'endpoint_override=self.get_endpoint(service_key)' | |
| 12:29:37 | mnaser | (before the **kwargs) | |
| 12:30:05 | mnaser | and it also fixes the issue-- CheckBarbicanIntegration CRITICAL: Unable to establish connection to http://foobar/secrets: HTTPConnectionPool(host='foobar', port=80): Max retries exceeded with url: /secrets (Caused by NewConnectionError(' |
|
| 12:32:22 | mordred | mnaser: whoops | |
| 12:32:51 | openstackgerrit | Monty Taylor proposed openstack/openstacksdk master: Honor endpoint_override for get_session_client https://review.openstack.org/565489 | |
| 12:35:20 | mordred | Shrews: ^^ if you get a sec | |
| 14:15:18 | mordred | Shrews: you may also enjoy https://review.openstack.org/549307 | |
| 14:15:40 | mordred | slaweq: ^^ you may also enjoy both of those patches (or you might not :) ) | |
| 14:15:54 | slaweq | mordred: looking | |
| 14:15:55 | slaweq | :) | |
| 14:57:35 | gtmanfred | mordred: ping | |
| 14:59:09 | mordred | gtmanfred: hi! I'm about to be on a phone call for an hour - but then I'll be here | |
| 14:59:52 | gtmanfred | awesome, thanks | |
| 15:00:49 | mordred | gtmanfred: however, that race condition you mention should not exist on a reasonably modern openstack - we use the create-floaing-ip-with-server-port approach rather than the 'create-floating-ip-then-attach-to-server' approach which suffers from the race condition mentioned | |
| 15:01:37 | mordred | if the cloud in quetsion does not expose the neutron endpoints needed to safely create floating ips in a multi-process or multi-threaded manner, we will fall back to create-and-attach | |
| 15:02:03 | mordred | (this is all just backgound info) | |
| 15:04:00 | gtmanfred | kk | |
| 15:04:55 | gtmanfred | lemme check what they are running, it is older | |
| 15:06:07 | mordred | yah- the pure nova api version of this is ... not great .. for the reasons listed in the story | |
| 15:10:07 | gtmanfred | mordred: he is using mitaka | |
| 15:12:06 | mordred | hrm. that should be new enough | |
| 15:17:44 | gtmanfred | I also think I tested with pike, and was running into the same problem. | |
| 15:18:05 | gtmanfred | what it seems like the problem is, is that we are trying to use floating ips that have already been allocated | |
| 15:18:30 | gtmanfred | and since multiple processes list all the available ips, and then pick one, multiple processes try to assign the same ip | |
| 15:19:38 | gtmanfred | this is why we went with `create and assign by default. if there are no free ips, log a warning that we are going to be not multiprocess safe, because we need to list available ones and pick one from a list` | |
| 15:23:08 | openstackgerrit | Hongbin Lu proposed openstack/openstacksdk master: Add 'port_details' to Floating IP https://review.openstack.org/533811 | |
| 15:23:39 | mordred | yah. multi-process re-use of pre-existing floating ips is basically unpossible - without some sort of shared locking/brokering system | |
| 15:24:32 | mordred | in nodepool (which does this process in many many parallel threads) - we make sure we delete floating ips when we delete servers - and also have a cleanup thread that deletes any unattached floating ips it finds | |
| 15:25:13 | mordred | since if the create-with-server-port method is being used, there is no such thing as a validly unattached floating ip | |
| 15:25:27 | gtmanfred | ok, i think what I am just going to do is change the default `reuse_ips` to False, and allow them to set it if they really really really really really really really really are dumb and want to use it | |
| 15:26:25 | openstackgerrit | XiaojueGuan proposed openstack/keystoneauth master: Trivial: Update pypi url to new url https://review.openstack.org/565418 | |
| 15:26:57 | mordred | gtmanfred: yah | |
| 15:27:05 | mordred | gtmanfred: so - that said - there is an option we can do | |
| 15:27:36 | mordred | (it's on my list already = it's just deep down there and there's a few big ticket items we need to likely get done first) | |
| 15:28:46 | mordred | task number 1 is to finish making the caching tier use dogpile.cache everywhere (servers, ports and floating ips are currently done manually if caching is enabled - and there are some tricky interactions as it relates to batched operations and thundering herd that we hvae to be careful about when fixing that) | |
| 15:29:27 | mordred | but once we've got that, then we should be able to add in a locking/multi-process aware TaskManager (similar to the multithreaded TaskManager that's in nodepool right now) | |
| 15:29:43 | mordred | and leverge dogpile for the shared locking | |
| 15:29:47 | gtmanfred | :+1: | |
| 15:30:06 | gtmanfred | Lemme know what I can help with, i need to figure out what to contribute to so I can go to berlin. | |
| 15:30:06 | mordred | so if someone then wants to do multi-process with reuse_ips there will be a story for it | |
| 15:30:37 | mordred | cool. I'll write up the above into a more consumable story that doesn't involve memories in my head and ping you with it | |
| 15:30:47 | gtmanfred | that would be great | |
| 15:30:54 | mordred | \o/ winning | |
| 15:34:23 | gtmanfred | other than that, the new shade cloud driver has gotten rave reviews, and everyone says it is much easier to configure and more reliable than what we had before | |
| 15:54:29 | openstackgerrit | Merged openstack/openstacksdk master: Drop bogus attributes from network port resource https://review.openstack.org/565217 | |
| 16:56:47 | mordred | gtmanfred: yay! | |
| 19:21:55 | gtmanfred | mordred: also, with the cache stuff for servers, it is about 10 times faster | |
| 19:23:34 | mordred | yah. the cache stuff is really important - and the other thing is also about being able to enable at least in-memory caching by default | |
| 19:56:27 | crunchengine | mordred: hello | |
| 19:57:02 | mordred | crunchengine: hiya | |
| 19:57:11 | crunchengine | I am listing active instances, but compute give me migrating instances too | |
| 19:57:27 | crunchengine | self.conn.compute.servers(host=self.hostname, all_tenants=True, status="ACTIVE") | |
| 19:58:12 | crunchengine | those instances are still active, from kvm point of view but the filter should be state consistent no ? | |
| 19:59:23 | mordred | I dunno about the kvm POV - but the filter from a REST API perspective only knows about the value of teh status field. doesn't migration status show up in vm_state? | |
| 19:59:38 | mordred | ah - sorry - progress | |
| 19:59:54 | mordred | nope. thinking out loud - sorry :) | |
| 20:01:09 | mordred | yah - I'd expect for the status of the vm to be 'MIGRATING' from looking at the api docs | |
| 20:01:24 | crunchengine | print("uuid: {} status: {} task_state: {} vm_state: {}".format(i.id, i.status, i.task_state, i.vm_state)) | |
| 20:01:30 | crunchengine | uuid: 5a562937-b0e2-4e67-8b2e-490cd44bbc3d status: MIGRATING task_state: migrating vm_state: active | |
| 20:01:38 | crunchengine | too much state ! | |
| 20:01:45 | mordred | right? | |
| 20:01:47 | crunchengine | okay thanks ! | |
| 20:03:35 | crunchengine | so naming is inconsistent too: return self.conn.compute.servers(host=self.hostname, all_tenants=True, status="ACTIVE") | |
| 20:03:48 | crunchengine | status => vm_state in lowercase... | |
| 21:58:31 | openstackgerrit | Merged openstack/keystoneauth master: Allow tuples and sets in interface list https://review.openstack.org/564495 | |
| #openstack-sdks - 2018-05-02 | |||
| 10:24:46 | rabel | hi there. does osc support assigning system roles? ( https://developer.openstack.org/api-ref/identity/v3/#system-role-assignments ) | |
| 11:15:03 | pooja_jadhav | cmurphy: Hi | |
| 11:45:12 | umbSublime | is the ressource.Body updated on a ressource when something changes on the object. For example if i check a Server.hypervisor_hostname property, and then migrate the server. Do Ineed to recreate my server object to get the new Server.hypervisor_hostname ? | |
| 13:52:20 | openstackgerrit | Merged openstack/os-client-config master: Replace guts with openstack.config https://review.openstack.org/549307 | |
| 15:03:06 | umbSublime | perhaps my question isn't clear, feel free to ask if so. | |
| 15:21:51 | elmiko | umbSublime: that seems like a nova question, but if you migrated the server through an openstack interface then it _should_ update the resource for you | |
| 15:22:18 | elmiko | if you did the migration outside of openstack, then i'm not sure what the behavrior would be | |
| 15:22:40 | elmiko | in general, we trust the resource servers to be the source of truth for information about the resources they manage | |
| 15:22:48 | elmiko | hope that helps | |
| 15:33:54 | openstackgerrit | Merged openstack/openstacksdk master: Honor endpoint_override for get_session_client https://review.openstack.org/565489 | |
| 15:35:43 | umbSublime | elmiko: What I mean is, if I create an sdk Server ressource, and then call it's 'migration' method. Once the migration is completed, can I check the new hypervisor with my existing Server ressource, or should I create a new one with the SDK | |
| 15:36:07 | umbSublime | to reflect those changes | |
| 15:43:02 | openstackgerrit | Merged openstack/openstacksdk master: Don't assume a full config dict https://review.openstack.org/564493 | |
| 15:44:25 | elmiko | umbSublime: i'm not sure about the sdk specifics, but i would imagine that at the minimum the Server uuid would be the same. not sure if that means you can use the resource or not. sorry, i misunderstood what you were asking. | |
| 15:44:49 | umbSublime | np elmiko thanks for giving it a shot :p | |
| 15:45:03 | elmiko | i would think though, that if the resource has a migration method, then you can continue to use the same resource object | |
| 15:45:19 | elmiko | presumably it is communicating with the openstack server on the backend | |
| 15:45:33 | umbSublime | I'd agree, (i'm still novice prrgrammer), but when looking at the code I see no mechanism that do that | |
| 15:45:41 | elmiko | ahh, gotcha | |
| 15:45:58 | elmiko | it might be safest to aquire a new server resource using the id of the old one | |
| 15:46:08 | elmiko | for maximum paranoia ;) | |
| 15:46:52 | elmiko | theoretically, on the backend side, the server resource id should be the same, even after migration | |
| 15:59:29 | umbSublime | elmiko: yah for sure, I was talking hypervisor_hostname, which should change after a migration | |
| 15:59:42 | umbSublime | in the meantime I agree with you i'll update my ressource :) | |