{"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-693009048", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 693009048, "node_id": "MDEyOklzc3VlQ29tbWVudDY5MzAwOTA0OA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-15T22:17:30Z", "updated_at": "2020-09-22T14:37:00Z", "author_association": "OWNER", "body": "Maybe instead of implementing `datasette.get()` and `datasette.post()` and `datasette.request()` and `datasette.stream()` I could instead have a nested object called `datasette.client` which is a preconfigured `AsyncClient` instance.\r\n\r\n```python\r\nresponse = await datasette.client.get(\"/\")\r\n```\r\nOr perhaps this should be a method in case I ever need to be able to `await` it:\r\n```python\r\nresponse = await (await datasette.client()).get(\"/\")\r\n```\r\nThis is a bit cosmetically ugly though, I'd rather avoid that if possible.\r\n\r\nMaybe I could get this working by returning an object from `.client()` which provides a `await obj.get()` method:\r\n```python\r\nresponse = await datasette.client().get(\"/\")\r\n```\r\nI don't think there's any benefit to that over `await datasette.client.get()` though.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-696769501", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 696769501, "node_id": "MDEyOklzc3VlQ29tbWVudDY5Njc2OTUwMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-22T14:45:49Z", "updated_at": "2020-09-22T14:45:49Z", "author_association": "OWNER", "body": "I put together a minimal prototype of this and it feels pretty good:\r\n```diff\r\ndiff --git a/datasette/app.py b/datasette/app.py\r\nindex 20aae7d..fb3bdad 100644\r\n--- a/datasette/app.py\r\n+++ b/datasette/app.py\r\n@@ -4,6 +4,7 @@ import collections\r\n import datetime\r\n import glob\r\n import hashlib\r\n+import httpx\r\n import inspect\r\n import itertools\r\n from itsdangerous import BadSignature\r\n@@ -312,6 +313,7 @@ class Datasette:\r\n         self._register_renderers()\r\n         self._permission_checks = collections.deque(maxlen=200)\r\n         self._root_token = secrets.token_hex(32)\r\n+        self.client = DatasetteClient(self)\r\n \r\n     async def invoke_startup(self):\r\n         for hook in pm.hook.startup(datasette=self):\r\n@@ -1209,3 +1211,25 @@ def route_pattern_from_filepath(filepath):\r\n \r\n class NotFoundExplicit(NotFound):\r\n     pass\r\n+\r\n+\r\n+class DatasetteClient:\r\n+    def __init__(self, ds):\r\n+        self.app = ds.app()\r\n+\r\n+    def _fix(self, path):\r\n+        if path.startswith(\"/\"):\r\n+            path = \"http://localhost{}\".format(path)\r\n+        return path\r\n+\r\n+    async def get(self, path, **kwargs):\r\n+        async with httpx.AsyncClient(app=self.app) as client:\r\n+            return await client.get(self._fix(path), **kwargs)\r\n+\r\n+    async def post(self, path, **kwargs):\r\n+        async with httpx.AsyncClient(app=self.app) as client:\r\n+            return await client.post(self._fix(path), **kwargs)\r\n+\r\n+    async def options(self, path, **kwargs):\r\n+        async with httpx.AsyncClient(app=self.app) as client:\r\n+            return await client.options(self._fix(path), **kwargs)\r\n```\r\nUsed like this in `ipython`:\r\n```\r\nIn [1]: from datasette.app import Datasette\r\n\r\nIn [2]: ds = Datasette([\"fixtures.db\"])\r\n\r\nIn [3]: (await ds.client.get(\"/-/config.json\")).json()\r\nOut[3]: \r\n{'default_page_size': 100,\r\n 'max_returned_rows': 1000,\r\n 'num_sql_threads': 3,\r\n 'sql_time_limit_ms': 1000,\r\n 'default_facet_size': 30,\r\n 'facet_time_limit_ms': 200,\r\n 'facet_suggest_time_limit_ms': 50,\r\n 'hash_urls': False,\r\n 'allow_facet': True,\r\n 'allow_download': True,\r\n 'suggest_facets': True,\r\n 'default_cache_ttl': 5,\r\n 'default_cache_ttl_hashed': 31536000,\r\n 'cache_size_kb': 0,\r\n 'allow_csv_stream': True,\r\n 'max_csv_mb': 100,\r\n 'truncate_cells_html': 2048,\r\n 'force_https_urls': False,\r\n 'template_debug': False,\r\n 'base_url': '/'}\r\n\r\nIn [4]: (await ds.client.get(\"/fixtures/facetable.json?_shape=array\")).json()\r\nOut[4]: \r\n[{'pk': 1,\r\n  'created': '2019-01-14 08:00:00',\r\n  'planet_int': 1,\r\n  'on_earth': 1,\r\n  'state': 'CA',\r\n  'city_id': 1,\r\n  'neighborhood': 'Mission',\r\n  'tags': '[\"tag1\", \"tag2\"]',\r\n  'complex_array': '[{\"foo\": \"bar\"}]',\r\n  'distinct_some_null': 'one'},\r\n {'pk': 2,\r\n  'created': '2019-01-14 08:00:00',\r\n  'planet_int': 1,\r\n  'on_earth': 1,\r\n  'state': 'CA',\r\n  'city_id': 1,\r\n  'neighborhood': 'Dogpatch',\r\n  'tags': '[\"tag1\", \"tag3\"]',\r\n  'complex_array': '[]',\r\n  'distinct_some_null': 'two'},\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-696769853", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 696769853, "node_id": "MDEyOklzc3VlQ29tbWVudDY5Njc2OTg1Mw==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-22T14:46:21Z", "updated_at": "2020-09-22T14:46:21Z", "author_association": "OWNER", "body": "This adds `httpx` as a dependency - I think I'm OK with that. I use it for testing in all of my plugins anyway.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-696774711", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 696774711, "node_id": "MDEyOklzc3VlQ29tbWVudDY5Njc3NDcxMQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-22T14:53:56Z", "updated_at": "2020-09-22T14:53:56Z", "author_association": "OWNER", "body": "How important is it to use `httpx.AsyncClient` with a context manager?\r\n\r\nhttps://www.python-httpx.org/async/#opening-and-closing-clients says:\r\n\r\n> Alternatively, use `await client.aclose()` if you want to close a client explicitly:\r\n> \r\n> ```\r\n> client = httpx.AsyncClient()\r\n> ...\r\n> await client.aclose()\r\n> ```\r\nThe `.aclose()` method has a comment saying \"Close transport and proxies\" - I'm not using proxies, so the relevant implementation seems to be a call to `await self._transport.aclose()` in https://github.com/encode/httpx/blob/f932af9172d15a803ad40061a4c2c0cd891645cf/httpx/_client.py#L1741-L1751\r\n\r\nThe transport I am using is a class called `ASGITransport` in https://github.com/encode/httpx/blob/master/httpx/_transports/asgi.py\r\n\r\nThe `aclose()` method on that class does nothing. So it looks like I can instantiate a client without bothering with the `async with httpx.AsyncClient` bit.", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-696775516", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 696775516, "node_id": "MDEyOklzc3VlQ29tbWVudDY5Njc3NTUxNg==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-22T14:55:10Z", "updated_at": "2020-09-22T14:55:10Z", "author_association": "OWNER", "body": "Even smaller `DatasetteClient` implementation:\r\n```python\r\nclass DatasetteClient:\r\n    def __init__(self, ds):\r\n        self._client = httpx.AsyncClient(app=ds.app())\r\n\r\n    def _fix(self, path):\r\n        if path.startswith(\"/\"):\r\n            path = \"http://localhost{}\".format(path)\r\n        return path\r\n\r\n    async def get(self, path, **kwargs):\r\n        return await self._client.get(self._fix(path), **kwargs)\r\n\r\n    async def post(self, path, **kwargs):\r\n        return await self._client.post(self._fix(path), **kwargs)\r\n\r\n    async def options(self, path, **kwargs):\r\n        return await self._client.options(self._fix(path), **kwargs)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-696776828", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 696776828, "node_id": "MDEyOklzc3VlQ29tbWVudDY5Njc3NjgyOA==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-22T14:57:13Z", "updated_at": "2020-09-22T14:57:13Z", "author_association": "OWNER", "body": "I may as well implement all of the HTTP methods supported by the `httpx` client:\r\n\r\n- get\r\n- options\r\n- head\r\n- post\r\n- put\r\n- patch\r\n- delete", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-696777886", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 696777886, "node_id": "MDEyOklzc3VlQ29tbWVudDY5Njc3Nzg4Ng==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-22T14:58:54Z", "updated_at": "2020-09-22T14:58:54Z", "author_association": "OWNER", "body": "```python\r\nclass DatasetteClient:\r\n    def __init__(self, ds):\r\n        self._client = httpx.AsyncClient(app=ds.app())\r\n\r\n    def _fix(self, path):\r\n        if path.startswith(\"/\"):\r\n            path = \"http://localhost{}\".format(path)\r\n        return path\r\n\r\n    async def get(self, path, **kwargs):\r\n        return await self._client.get(self._fix(path), **kwargs)\r\n\r\n    async def options(self, path, **kwargs):\r\n        return await self._client.options(self._fix(path), **kwargs)\r\n\r\n    async def head(self, path, **kwargs):\r\n        return await self._client.head(self._fix(path), **kwargs)\r\n\r\n    async def post(self, path, **kwargs):\r\n        return await self._client.post(self._fix(path), **kwargs)\r\n\r\n    async def put(self, path, **kwargs):\r\n        return await self._client.put(self._fix(path), **kwargs)\r\n\r\n    async def patch(self, path, **kwargs):\r\n        return await self._client.patch(self._fix(path), **kwargs)\r\n\r\n    async def delete(self, path, **kwargs):\r\n        return await self._client.delete(self._fix(path), **kwargs)\r\n```", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null}
{"html_url": "https://github.com/simonw/datasette/issues/943#issuecomment-696778735", "issue_url": "https://api.github.com/repos/simonw/datasette/issues/943", "id": 696778735, "node_id": "MDEyOklzc3VlQ29tbWVudDY5Njc3ODczNQ==", "user": {"value": 9599, "label": "simonw"}, "created_at": "2020-09-22T15:00:13Z", "updated_at": "2020-09-22T15:00:39Z", "author_association": "OWNER", "body": "Am I going to rewrite ALL of my tests to use this instead? It would clean up a lot of test code, at the cost of quite a bit of work.\r\n\r\nIt would make for much neater plugin tests too, and neater testing documentation: https://docs.datasette.io/en/stable/testing_plugins.html", "reactions": "{\"total_count\": 0, \"+1\": 0, \"-1\": 0, \"laugh\": 0, \"hooray\": 0, \"confused\": 0, \"heart\": 0, \"rocket\": 0, \"eyes\": 0}", "issue": {"value": 681375466, "label": "await datasette.client.get(path) mechanism for executing internal requests"}, "performed_via_github_app": null}