html_url,issue_url,id,node_id,user,user_label,created_at,updated_at,author_association,body,reactions,issue,issue_label,performed_via_github_app
https://github.com/dogsheep/dogsheep-photos/issues/19#issuecomment-624406285,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/19,624406285,MDEyOklzc3VlQ29tbWVudDYyNDQwNjI4NQ==,9599,simonw,2020-05-06T02:10:03Z,2020-05-06T02:10:03Z,MEMBER,"Most annoying part of this is the difficulty of actually showing a photo.
Maybe I need to run a local proxy that I can link to? A custom Datasette plugin perhaps?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613002220,apple-photos command should work even if upload has not run,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-624408220,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,624408220,MDEyOklzc3VlQ29tbWVudDYyNDQwODIyMA==,9599,simonw,2020-05-06T02:18:47Z,2020-05-06T02:18:47Z,MEMBER,"The `apple_photos` table has an indexed `uuid` column and a `path` column which stores the full path to that photo file on disk.
I can write a custom Datasette plugin which takes the `uuid` from the URL, looks up the path, then serves up a thumbnail of the jpeg or heic image file.
I'll prototype this is a one-off plugin first, then package it on PyPI for other people to install.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-624408370,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,624408370,MDEyOklzc3VlQ29tbWVudDYyNDQwODM3MA==,9599,simonw,2020-05-06T02:19:27Z,2020-05-06T02:19:27Z,MEMBER,"The plugin can be generalized: it can be configured to know how to take the URL path, look it up in ANY table (via a custom SQL query) to get a path on disk and then serve that.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/dogsheep/dogsheep-photos/issues/20#issuecomment-624408738,https://api.github.com/repos/dogsheep/dogsheep-photos/issues/20,624408738,MDEyOklzc3VlQ29tbWVudDYyNDQwODczOA==,9599,simonw,2020-05-06T02:21:05Z,2020-05-06T02:21:32Z,MEMBER,"Here's rendering code from my hacked-together not-yet-released S3 image proxy:
```python
from starlette.responses import Response
from PIL import Image, ExifTags
import pyheif
for ORIENTATION_TAG in ExifTags.TAGS.keys():
if ExifTags.TAGS[ORIENTATION_TAG] == ""Orientation"":
break
...
# Load it into Pillow
if ext == ""heic"":
heic = pyheif.read_heif(image_response.content)
image = Image.frombytes(mode=heic.mode, size=heic.size, data=heic.data)
else:
image = Image.open(io.BytesIO(image_response.content))
# Does EXIF tell us to rotate it?
try:
exif = dict(image._getexif().items())
if exif[ORIENTATION_TAG] == 3:
image = image.rotate(180, expand=True)
elif exif[ORIENTATION_TAG] == 6:
image = image.rotate(270, expand=True)
elif exif[ORIENTATION_TAG] == 8:
image = image.rotate(90, expand=True)
except (AttributeError, KeyError, IndexError):
pass
# Resize based on ?w= and ?h=, if set
width, height = image.size
w = request.query_params.get(""w"")
h = request.query_params.get(""h"")
if w is not None or h is not None:
if h is None:
# Set h based on w
w = int(w)
h = int((float(height) / width) * w)
elif w is None:
h = int(h)
# Set w based on h
w = int((float(width) / height) * h)
w = int(w)
h = int(h)
image.thumbnail((w, h))
# ?bw= converts to black and white
if request.query_params.get(""bw""):
image = image.convert(""L"")
# ?q= sets the quality - defaults to 75
quality = 75
q = request.query_params.get(""q"")
if q and q.isdigit() and 1 <= int(q) <= 100:
quality = int(q)
# Output as JPEG or PNG
output_image = io.BytesIO()
image_type = ""JPEG""
kwargs = {""quality"": quality}
if image.format == ""PNG"":
image_type = ""PNG""
kwargs = {}
image.save(output_image, image_type, **kwargs)
return Response(
output_image.getvalue(),
media_type=""image/jpeg"",
headers={""cache-control"": ""s-maxage={}, public"".format(365 * 24 * 60 * 60)},
)
```","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613006393,Ability to serve thumbnailed Apple Photo from its place on disk,
https://github.com/simonw/datasette/issues/760#issuecomment-624729459,https://api.github.com/repos/simonw/datasette/issues/760,624729459,MDEyOklzc3VlQ29tbWVudDYyNDcyOTQ1OQ==,9599,simonw,2020-05-06T15:47:44Z,2020-05-06T15:47:44Z,OWNER,"`select * from pragma_table_info(tablename);` is currently disallowed for user-provided queries via a regex restriction - but could help here too.
https://github.com/simonw/datasette/blob/d349d57cdf3d577afb62bdf784af342a4d5be660/datasette/utils/__init__.py#L174","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613422636,Way of seeing full schema for a database,
https://github.com/simonw/datasette/issues/761#issuecomment-624766424,https://api.github.com/repos/simonw/datasette/issues/761,624766424,MDEyOklzc3VlQ29tbWVudDYyNDc2NjQyNA==,9599,simonw,2020-05-06T16:54:38Z,2020-05-06T17:01:02Z,OWNER,"I could allow-list some other useful `pragma_x` tables too.
SQLite calls these ""pragma functions"" - documented here: https://www.sqlite.org/pragma.html#pragfunc
They sound safe:
> Table-valued functions exist only for PRAGMAs that return results and that have no side-effects. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613467382,Allow-list pragma_table_info(tablename) and similar,
https://github.com/simonw/datasette/issues/761#issuecomment-624767466,https://api.github.com/repos/simonw/datasette/issues/761,624767466,MDEyOklzc3VlQ29tbWVudDYyNDc2NzQ2Ng==,9599,simonw,2020-05-06T16:56:40Z,2020-05-06T16:57:03Z,OWNER,"The rationale for blocking `pragma` entirely from statements is that it can be used to change the state of the SQLite database, e.g. from https://www.sqlite.org/pragma.html :
```
PRAGMA schema.application_id;
PRAGMA schema.application_id = integer ;
```
That second line is unsafe. I don't think it's possible to use the `pragma_table_x` variants to make writes in this way.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613467382,Allow-list pragma_table_info(tablename) and similar,
https://github.com/simonw/datasette/issues/761#issuecomment-624768744,https://api.github.com/repos/simonw/datasette/issues/761,624768744,MDEyOklzc3VlQ29tbWVudDYyNDc2ODc0NA==,9599,simonw,2020-05-06T16:59:01Z,2020-05-06T16:59:01Z,OWNER,"Maybe use a negative lookahead assertion? https://docs.python.org/3/library/re.html#index-20
> `(?!...)`
>
> Matches if `...` doesn’t match next. This is a negative lookahead assertion. For example, `Isaac (?!Asimov)` will match 'Isaac ' only if it’s not followed by 'Asimov'.","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613467382,Allow-list pragma_table_info(tablename) and similar,
https://github.com/simonw/datasette/issues/761#issuecomment-624774928,https://api.github.com/repos/simonw/datasette/issues/761,624774928,MDEyOklzc3VlQ29tbWVudDYyNDc3NDkyOA==,9599,simonw,2020-05-06T17:11:15Z,2020-05-06T17:11:15Z,OWNER,"For the moment I'll allow-list the following:
* `pragma_database_list`
* `pragma_foreign_key_list`
* `pragma_function_list`
* `pragma_index_info`
* `pragma_index_list`
* `pragma_index_xinfo`
* `pragma_page_count`
* `pragma_max_page_count`
* `pragma_page_size`
* `pragma_schema_version`
* `pragma_table_info`
* `pragma_table_xinfo`
","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613467382,Allow-list pragma_table_info(tablename) and similar,
https://github.com/simonw/datasette/issues/761#issuecomment-624782775,https://api.github.com/repos/simonw/datasette/issues/761,624782775,MDEyOklzc3VlQ29tbWVudDYyNDc4Mjc3NQ==,9599,simonw,2020-05-06T17:26:05Z,2020-05-06T17:26:05Z,OWNER,"Some demos:
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_database_list%28%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_foreign_key_list%28%27complex_foreign_keys%27%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_function_list%28%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_index_info%28%27idx_compound_three_primary_keys_content%27%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_index_list%28%27compound_three_primary_keys%27%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_index_xinfo%28%27idx_compound_three_primary_keys_content%27%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_page_count%28%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_max_page_count%28%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_page_size%28%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_schema_version%28%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_table_info%28%27complex_foreign_keys%27%29
* https://latest.datasette.io/fixtures?sql=select+*+from+pragma_table_xinfo%28%27complex_foreign_keys%27%29","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613467382,Allow-list pragma_table_info(tablename) and similar,
https://github.com/simonw/datasette/issues/761#issuecomment-624783996,https://api.github.com/repos/simonw/datasette/issues/761,624783996,MDEyOklzc3VlQ29tbWVudDYyNDc4Mzk5Ng==,9599,simonw,2020-05-06T17:28:20Z,2020-05-06T17:28:20Z,OWNER,"Interestingly https://latest.datasette.io/fixtures?sql=select+*+from+pragma_function_list() doesn't work, when it DOES work on my laptop.
`latest.datasette.io` currently runs SQLite `3.27.2` while my laptop runs `3.31.1`
https://www.sqlite.org/changes.html#version_3_30_0 says that as-of 3.30.0:
> The PRAGMA function_list, PRAGMA module_list, and PRAGMA pragma_list commands are now enabled in all builds by default. Disable them using -DSQLITE_OMIT_INTROSPECTION_PRAGMAS. ","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613467382,Allow-list pragma_table_info(tablename) and similar,
https://github.com/simonw/datasette/issues/760#issuecomment-624787678,https://api.github.com/repos/simonw/datasette/issues/760,624787678,MDEyOklzc3VlQ29tbWVudDYyNDc4NzY3OA==,9599,simonw,2020-05-06T17:35:05Z,2020-05-06T17:35:05Z,OWNER,Potential recipe in https://github.com/simonw/til/blob/master/sqlite/list-all-columns-in-a-database.md,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613422636,Way of seeing full schema for a database,
https://github.com/simonw/datasette/issues/761#issuecomment-624790887,https://api.github.com/repos/simonw/datasette/issues/761,624790887,MDEyOklzc3VlQ29tbWVudDYyNDc5MDg4Nw==,9599,simonw,2020-05-06T17:41:21Z,2020-05-06T17:41:21Z,OWNER,More demos here: https://github.com/simonw/til/blob/master/sqlite/list-all-columns-in-a-database.md,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613467382,Allow-list pragma_table_info(tablename) and similar,
https://github.com/simonw/datasette/issues/759#issuecomment-624795695,https://api.github.com/repos/simonw/datasette/issues/759,624795695,MDEyOklzc3VlQ29tbWVudDYyNDc5NTY5NQ==,9599,simonw,2020-05-06T17:50:57Z,2020-05-06T17:52:07Z,OWNER,"This was a deliberate change from #651.
The `_search_colname=` alternative argument for doing this still works - compare these two:
* https://latest.datasette.io/fixtures/searchable?_search=dog
* https://latest.datasette.io/fixtures/searchable?_search_tex1=dog
If you want to use advanced search syntax on those pages you can do so using the `&_searchmode=raw` option - I added better documentation for this the other day: https://datasette.readthedocs.io/en/latest/full_text_search.html#advanced-sqlite-search-queries
Example: https://latest.datasette.io/fixtures/searchable?_search=text1:dog&_searchmode=raw","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612673948,fts search on a column doesn't work anymore due to escape_fts,
https://github.com/simonw/datasette/issues/758#issuecomment-624796685,https://api.github.com/repos/simonw/datasette/issues/758,624796685,MDEyOklzc3VlQ29tbWVudDYyNDc5NjY4NQ==,9599,simonw,2020-05-06T17:52:54Z,2020-05-06T17:52:54Z,OWNER,Thanks for the suggestion! I'll add this.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612382643,Question: Access to immutable database-path,
https://github.com/simonw/datasette/issues/758#issuecomment-624797119,https://api.github.com/repos/simonw/datasette/issues/758,624797119,MDEyOklzc3VlQ29tbWVudDYyNDc5NzExOQ==,9599,simonw,2020-05-06T17:53:46Z,2020-05-06T17:53:46Z,OWNER,It's interesting to hear from someone who's using this feature - I'm considering moving it out into a plugin #647.,"{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612382643,Question: Access to immutable database-path,
https://github.com/simonw/datasette/issues/757#issuecomment-624798182,https://api.github.com/repos/simonw/datasette/issues/757,624798182,MDEyOklzc3VlQ29tbWVudDYyNDc5ODE4Mg==,9599,simonw,2020-05-06T17:55:50Z,2020-05-06T17:55:50Z,OWNER,"I'll definitely get that out this week!
For the moment a trick I often use is to put a URL to the most recent commit in my `requirements.txt` - e.g. https://github.com/simonw/datasette/archive/0784f2ef9d3ff6dd9df05f54cb51de29a6d11764.zip
This should be safe because nothing lands on Datasette master without the full unit test suite passing. But you're right, there's a bunch of stuff now that needs to go out in a release: https://github.com/simonw/datasette/compare/0.40...0784f2ef9d3ff6dd9df05f54cb51de29a6d11764","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612378203,Question: Any fixed date for the release with the uft8-encoding fix?,
https://github.com/simonw/datasette/issues/757#issuecomment-624798540,https://api.github.com/repos/simonw/datasette/issues/757,624798540,MDEyOklzc3VlQ29tbWVudDYyNDc5ODU0MA==,9599,simonw,2020-05-06T17:56:34Z,2020-05-06T17:56:34Z,OWNER,Actually I'm going to put that release out today. I was hoping to finish #698 first but that shouldn't delay those other features any longer.,"{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612378203,Question: Any fixed date for the release with the uft8-encoding fix?,
https://github.com/simonw/datasette/issues/757#issuecomment-624821090,https://api.github.com/repos/simonw/datasette/issues/757,624821090,MDEyOklzc3VlQ29tbWVudDYyNDgyMTA5MA==,9599,simonw,2020-05-06T18:41:29Z,2020-05-06T18:41:29Z,OWNER,"OK, I just released 0.41 with that and a bunch of other stuff: https://datasette.readthedocs.io/en/latest/changelog.html#v0-41","{""total_count"": 1, ""+1"": 1, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612378203,Question: Any fixed date for the release with the uft8-encoding fix?,
https://github.com/simonw/datasette/issues/759#issuecomment-624860451,https://api.github.com/repos/simonw/datasette/issues/759,624860451,MDEyOklzc3VlQ29tbWVudDYyNDg2MDQ1MQ==,133845,Krazybug,2020-05-06T20:03:01Z,2020-05-06T20:04:42Z,NONE,"Thank you. Now it's ok with the url
http://localhost:8001/index/summary?_search=language%3Aeng&_sort=title&_searchmode=raw
But I'm not able to manage it in the metadata file. Here is mine (note that the sort column is taken into account)
Here it is:
```
{
""databases"": {
""index"": {
""tables"": {
""summary"": {
""sort"": ""title"",
""searchmode"": ""raw""
}
}
}
}
}
```
Any idea ?","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",612673948,fts search on a column doesn't work anymore due to escape_fts,
https://github.com/simonw/datasette/issues/760#issuecomment-624949809,https://api.github.com/repos/simonw/datasette/issues/760,624949809,MDEyOklzc3VlQ29tbWVudDYyNDk0OTgwOQ==,9599,simonw,2020-05-06T23:49:06Z,2020-05-06T23:49:06Z,OWNER,"```sql
select
sqlite_master.name as table_name,
table_info.*
from
sqlite_master
join pragma_table_info(sqlite_master.name) as table_info
order by
sqlite_master.name,
table_info.cid
```
https://latest.datasette.io/fixtures?sql=select%0D%0A++sqlite_master.name+as+table_name%2C%0D%0A++table_info.*%0D%0Afrom%0D%0A++sqlite_master%0D%0A++join+pragma_table_info%28sqlite_master.name%29+as+table_info%0D%0Aorder+by%0D%0A++sqlite_master.name%2C%0D%0A++table_info.cid","{""total_count"": 0, ""+1"": 0, ""-1"": 0, ""laugh"": 0, ""hooray"": 0, ""confused"": 0, ""heart"": 0, ""rocket"": 0, ""eyes"": 0}",613422636,Way of seeing full schema for a database,