The AI Front Page

Search Edition

Search: python

30 stories from 8 sources across 8 topics.

Stories

30

Sources

8

Topics

8

For You lens

22 stories in this edition match your reader profile.

Reader signals

3

Searches

0

Matches

22

Top score

129

Tune For You

Search Intent

python

This query becomes a recent For You signal, so matching stories can move up on the next personalized pass.

For YouReader CenterPromote Search

Edition Index

Topic, entity, and source map

Topics

Open Source13 storiesAgents3 storiesFunding3 storiesModels3 storiesResearch3 storiesSecurity3 stories

Entities

Claude7 storiesAnthropic4 storiesOpenAI4 storiesGoogle3 storiesHugging Face2 storiesarXiv1 story

Sources

GitHub Trending AI11 storiesSimon Willison LLMs9 storiesHacker News AI4 storiesAWS Machine Learning Blog2 storiesarXiv AI/ML1 storyLangChain Blog1 story

Simon Willison LLMs / 11:56 PM

Generating running routes with GPT-6 Astra and ChatGPT Work

Here's a neat thing I had ChatGPT Work with GPT-6 Astra (Max) do this morning: I live at <my address>. Figure out 5K and 10K running routes from me that loop from my house. Use OSM data. It worked for 27 minutes and produced exactly what I'd asked for, as both an embedded visualization and downloadable GPX file and GeoJSON files. Here's that 5K route: When I asked it how it had created the route, it replied: I used Nominatim to locate the address and Overpass to download local OpenStreetMap roads and trails , then calculated the loops locally. Frustratingly, the actual code it ran and exact details of what it did weren't visible to me in the ChatGPT UI. I see this lack of transparency is an anti-feature. By the time I thought to ask for a copy of the Python code it had used, ChatGPT was unable to provide it. This appears to be because the thread had been compacted. I think any LLM system that uses compaction needs to both preserve the pre-compacted text and make that text available via agent tool calls, to protect against this kind of problem. As for displaying the map to me, that used the visualize skill . It created a file called /workspace/el-granada-5k-share.html to embed directly into the ChatGPT UI. Here's a copy of that HTML , which starts like this: < div id =" eg-share-loop " > < div class =" viz-row " > < h3 > El Granada harbor loop </ h3 > < span class =" text-small " > 5.1 km </ span > </ div > < div id =" eg-share-stage " > </ div > < div class =" text-small text-muted " > Map data © < a href =" https://www.openstreetmap.org/copyright " target =" _blank " rel =" noopener " > OpenStreetMap contributors </ a > </ div > < style > # eg-share-loop { width : 100 % ; } # eg-share-loop # eg-share-stage { width : 100 % ; margin : 8 px 0 ; } # eg-share-loop . eg-share-map { display : block; width : 100 % ; touch-action : none; } # eg-share-loop . eg-share-map text { fill : var ( --foreground ); font-size : 12 px ; font-weight : 400 ; } # eg-share-loop . eg-share-label { paint-order : stroke; stroke : var ( --background ); stroke-width : 3 px ; stroke-linejoin : round; } </ style > < script type =" application/json " id =" eg-share-data " > { "route" : { "type" : "LineString" , "coordinates" : [ [ - 122.467425 , 37.4997753 ] . . . </ script > < script src =" https://cdn.jsdelivr.net/npm/[email protected]/dist/d3.min.js " > </ script > < script > (() => { const root=document.getElementById('eg-share-loop'); The <script type="application/json"> element contains the full geometry needed to render both the running route and the map itself, using D3, which is loaded from an allow-listed CDN location described in this section of the visualize skill : External resources The CSP allows only cdnjs.cloudflare.com , esm.sh , cdn.jsdelivr.net , unpkg.com , fonts.googleapis.com , fonts.gstatic.com , and fonts.bunny.net . Other origins are blocked and fail silently. Tags: geospatial , ai , d3 , openai , generative-ai , chatgpt , llms , skills , gpt-6-astra

ReadSource

Simon Willison LLMs / 3:51 PM

Using Blender with coding agents on macOS

TIL: Using Blender with coding agents on macOS I've been having fun with Blender in ChatGPT Codex on my Mac recently. Getting it to work with coding agents is really easy: install the full Mac application from blender.org and run a prompt like this: Use the already install /Applications/Blender to render a scene of a pelican riding a bicycle In this case I followed that up with these two prompts: OK add a background and a lot of flair Then: OK make it a whole lot better And got this image, generated using Blender's Python API : This was covered by my existing Codex subscription, but according to AgentsView it would have cost $4.24 at API prices for gpt-6-astra . Tags: ai , generative-ai , llms , blender , pelican-riding-a-bicycle , coding-agents , gpt-6-astra

ReadSource

Simon Willison LLMs / 11:16 PM

smolmachines / smolvm as a sandbox for untrusted Python & JavaScript

Research: smolmachines / smolvm as a sandbox for untrusted Python & JavaScript I tasked Claude Fable 5 running in Claude Code for web with the following research task: Put https://smolmachines.com through its paces as a fast secure sandbox. Explore what it would take to use this to run untrusted Python and JavaScript code in a way that is limited in what RAM and CPU time it can take up (protection against "while true") with no network access and filesystem access only to designated files Goal is to be able to use this to execute user-provided tasks for things like data transformations It quickly ran into a problem: the Claude Code for web environment can't run smol machines . Quoting the notes it wrote : This Claude Code container: Linux 6.18.5-fc-v20 (itself a Firecracker guest), 4 vCPU, 15GB RAM. No /dev/kvm, no vmx/svm CPU flags → no nested virt. smolvm machine run fails as expected: "kvm not available". Plan B: GitHub Actions ubuntu runners DO expose /dev/kvm → run the real test battery via a temporary workflow on this branch, collect logs, remove workflow in final commit. And Plan B is what it did , installing smolvm and running these tests directly in a GitHub Actions runner against that branch. That was a creative solution to the environmental limits posed by Claude Code for web. Another example of Fable being relentlessly proactive . Tags: research , sandboxing , ai , github-actions , generative-ai , llms , coding-agents , claude-mythos-fable

ReadSource

Simon Willison LLMs / 7:37 PM

llm-gemini 0.33

Release: llm-gemini 0.33 It's been a while since the last llm-gemini release. This version of the plugin adds support for today's Gemini 3.7 Flash release, plus gemini-3.6-flash , gemini-3.5-flash-lite and two embedding models gemini-embedding-2 and gemini-embedding-001 . The plugin is also upgraded for compatibility with LLM 0.32, which means you can now see reasoning traces and you can also enable server-side tools using this pattern: llm -m gemini-3.7-flash -T CodeExecution \ 'use python to calculate (factorial of 13) * 3' I had Gemini 3.7 Flash draw me some pelicans riding bicycles at high, medium, and low thinking efforts (minimal, which was an option in 3.6 Flash, has been removed in 3.7.) Here's the high level one, which is pretty great: Update 14th August 2026 : I had originally said that the SVG rendered incorrectly in Chrome and Firefox, and blamed Gemini 3.7 Flash for producing invalid SVG. That was entirely incorrect: the rendering glitch was my fault, caused by a bug In my rendering tool . I've now fixed that bug. Tags: google , ai , generative-ai , llms , llm , gemini , pelican-riding-a-bicycle , llm-release

ReadSource

Simon Willison LLMs / 11:13 PM

Stateless MCP has recaptured my interest (and inspired mcp-explorer and datasette-mcp)

Tuesday was Stateless MCP day - the rollout of MCP 2.0, or the 2026-07-28 Model Context Protocol specification to use the more formal but less memorable name. This is the most significant change to the MCP spec since it first launched, and has also served to reignite my personal interest in the protocol. For background: MCP is the Model Context Protocol, which describes a standard way to expose new tools to LLM-powered agent frameworks. It was introduced by Anthropic back in November 2024 , had a huge spike of interest through much of 2025, and then became somewhat eclipsed by Skills (another Anthropic invention) when it became apparent that an agent harness with access to a terminal and curl could do most of what MCP did in a more flexible way. I wrote about that in my review of 2025 . I'm coming back around to MCP now. Giving an agent a shell environment with the ability to access the internet is fraught with risk , and requires a strong model that is capable of effectively driving such an environment. MCP tools are easier to audit and control, and simple enough that smaller models that run on a laptop can still drive them reasonably well. The new stateless MCP specification also greatly decreases the complexity of implementing both clients and servers for the protocol. I built three of those this week! What's easier with stateless MCP The best demonstration of the difference between stateful and stateless MCP is in this May 21st blog post that introduced the RC for the new specification. It included a clear before-and-after example. The older stateful MCP (I'm going to call it "legacy MCP") required two HTTP requests - the first to initialize a session and obtain a Mcp-Session-Id , and the second to actually call the tool: POST /mcp HTTP/1.1 Content-Type: application/json { "jsonrpc": "2.0", "id": 1, "method": "initialize", "params": { "protocolVersion": "2025-11-25", "capabilities": { }, "clientInfo": { "name": "my-app", "version": "1.0" } } } POST /mcp HTTP/1.1 Mcp-Session-Id: 1868a90c-3a3f-4f5b Content-Type: application/json { "jsonrpc": "2.0", "id": 2, "method": "tools/call", "params": { "name": "search", "arguments": { "q": "otters" } } } The new stateless way uses a single HTTP request which looks like this: POST /mcp HTTP/1.1 MCP-Protocol-Version: 2026-07-28 Mcp-Method: tools/call Mcp-Name: search Content-Type: application/json { "jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": { "name": "search", "arguments": { "q": "otters" }, "_meta": { "io.modelcontextprotocol/clientInfo": { "name": "my-app", "version": "1.0" } } } } This is so much cleaner from both a client- and server-side implementation perspective. It's also a better fit for building scalable web applications, since now you don't need to maintain server-side state to keep track of those session IDs, or worry about routing the same session to the same backend machine. mcp-explorer I couldn't find a great CLI tool for interactively probing an MCP server, so I had Codex help build my own. mcp-explorer is the result. It's a stateless Python CLI tool, so you don't even need to install it to try it out - it works with uvx like this: uvx mcp-explorer list https://agentic-mermaid.dev/mcp This queries Ade Oshineye's agentic-mermaid.dev demo MCP. The above command returns the following list of tools: execute(code: string, timeoutMs?: integer) - Execute Mermaid SDK code Run JavaScript in an isolated sandbox; return a value. describe_sdk(family: string, detail?: string) - Describe Mermaid SDK operations Return version-matched mutation operations for one diagram family. render_svg(source: string, options?: object) - Render Mermaid as SVG Render a Mermaid source string to themeable SVG. Returns { ok, svg }. render_ascii(source: string, useAscii?: boolean, targetWidth?: integer, options?: object) - Render Mermaid as text Render a Mermaid source string to text. Returns { ok, text }. render_png(source: string, scale?: number, background?: string, fitTo?: object, options?: object) - Render Mermaid as PNG Rasterize a Mermaid source string to PNG. Returns { ok, png_base64 }. ... Then to inspect a tool: uvx mcp-explorer inspect render_svg This outputs a whole bunch of information, including the JSON schema of the inputs and outputs. To call that tool and pass arguments to it: uvx mcp-explorer call \ https://agentic-mermaid.dev/mcp \ render_svg \ -a source ' graph TD; A-->B ' \ -a options ' {"padding":24} ' Which returns: {"ok":true,"svg":"<svg xmlns=\"http://www.w3.org/2000/svg\" width=... To get just the raw SVG try adding | jq .svg -r to that command. I got back this image : There are a few more commands in the README, but you get the general idea. I find building CLI tools like this to be a really productive way to get familiar with a specification, even if an agent writes most of the actual code. datasette-mcp The second project is datasette-mcp , a Datasette plugin which adds a /-/mcp endpoint to any Datasette instance. This is probably the fourth time I've tried building this plugin, but thanks to the new stateless MCP specification I finally have a version that feels good to release. It provides just three tools: list_databases() , get_database_schema(database_name) , and execute_sql(database_name, sql) . They do exactly what you would expect them to do - though execute_sql() is read-only for the moment. Wire these into an agent, or a chat tool like ChatGPT or Claude, and they'll gain the ability to run SQL queries against your hosted Datasette instance. So far I'm running it on the Datasette mirror of my blog, at datasette.simonwillison.net/-/mcp . It took a bit of fiddling to figure out how to attach that to ChatGPT and Claude, but I got there in the end. Here's a new TIL showing exactly how to do that. Here's a shared Claude session where I asked it: list tables in simonwillison.net And then: what has Simon said recently about MCP? It ran 7 separate SQL queries to figure out the answer. llm-mcp-client My LLM tool is long overdue for an official MCP integration. The new alpha llm-mcp-client plugin is my attempt at exactly that: llm install llm-mcp-client llm -T ' MCP("https://datasette.simonwillison.net/-/mcp") ' ' count the notes ' Here's the output (including reasoning trace, I'm using LLM 0.32rc2 ): Considering note count I see the question "count the notes" is probably asking me to tally up blog notes. It could also mean published notes or drafts, so there's some ambiguity there. I'll need to figure out the total number of notes, likely by querying the count for both published notes and drafts to get a clear answer. Let's execute that count! There are 151 notes . And the output of llm logs for that prompt. Once this is fully baked, I'm considering bringing it directly into LLM core. I'm excited to experiment with MCP in Datasette Agent and llm-coding-agent as well. MCP is a safer way to build with agents A few months after MCP was first released, I wrote Model Context Protocol has prompt injection security problems , where I noted that the pattern of having end users mix and match tools pushed responsibility for avoiding data exfiltration attacks out to the users themselves. I hadn't coined the Lethal Trifecta yet, but that was absolutely what I had in mind. Then general agents with arbitrary shell and curl access came along, and that's so much harder to keep secure! Something I've come to appreciate about MCP is that it's much easier to reason about agent capabilities and what might go wrong than with arbitrary command execution in an open network environment - the default for most of today's general and coding agent tools. I plan to lean into MCP a whole lot more when I'm building sensitive applications on top of LLMs. Tags: projects , ai , datasette , mermaid , generative-ai , llms , llm , anthropic , model-context-protocol

ReadSource

Simon Willison LLMs / 2:22 PM

Nativ: Run AI models locally on your Mac

Nativ: Run AI models locally on your Mac Prince Canuma is the developer behind the excellent MLX-VLM Python library for running vision-LLMs using MLX on a Mac. I'm really excited about his new project, which wraps MLX in a full macOS desktop application. It's similar in shape to LM Studio, providing both a chat interface and a localhost API server for accessing models. The app picked up MLX models I had already tried that were present in my Hugging Face cache directory, which was a nice touch. Via Hacker News Tags: macos , python , ai , generative-ai , local-llms , llms , mlx , prince-canuma

ReadSource

Simon Willison LLMs / 7:32 PM

sqlite-utils 4.0, now with database schema migrations

This morning I released sqlite-utils 4.0 , the 124th release of that project and the first major version bump since 3.0 in November 2020. In addition to some small but significant breaking changes (described in this upgrade guide ), this version introduces three major features: database migrations , nested transactions (via a new db.atomic() method), and support for compound foreign keys . Database schema migrations using sqlite-utils Schema migrations define a sequence of changes to be made to a SQLite database, plus a mechanism for tracking which migrations have been applied and applying any that are found to be pending. Migrations are defined in Python files using the sqlite-utils Python library , which includes a powerful table.transform() method providing enhanced alter table capabilities that are not supported by SQLite's ALTER TABLE statement. ( table.transform() implements the pattern recommended by the SQLite documentation - create a new temporary table with the new schema, copy across the data, then drop the old table and rename the temporary one in its place.) Here's an example migration file which creates a table called creatures , adds an additional column to it in a second step, then changes the types of two of the columns in a third: from sqlite_utils import Migrations migrations = Migrations ( "creatures" ) @ migrations () def create_table ( db ): db [ "creatures" ]. create ( { "id" : int , "name" : str , "species" : str }, pk = "id" , ) @ migrations () def add_weight ( db ): db [ "creatures" ]. add_column ( "weight" , float ) @ migrations () def change_column_types ( db ): db [ "creatures" ]. transform ( types = { "species" : int , "weight" : str }) Save that as migrations.py and run it against a fresh database like this: uvx sqlite-utils migrate data.db migrations.py Then if you check the schema of that database: uvx sqlite-utils schema data.db You'll see this SQL: CREATE TABLE " _sqlite_migrations " ( " id " INTEGER PRIMARY KEY , " migration_set " TEXT , " name " TEXT , " applied_at " TEXT ); CREATE UNIQUE INDEX " idx__sqlite_migrations_migration_set_name " ON " _sqlite_migrations " ( " migration_set " , " name " ); CREATE TABLE " creatures " ( " id " INTEGER PRIMARY KEY , " name " TEXT , " species " INTEGER , " weight " TEXT ); The _sqlite_migrations table is used to keep track of which migration functions have been run. The creatures table above is the schema after all three migrations have been applied. To see a list of migrations, both pending and applied, run this: uvx sqlite-utils migrate data.db migrations.py --list Output: Migrations for: creatures Applied: create_table - 2026-07-07 17:58:41.360051+00:00 add_weight - 2026-07-07 17:58:41.360608+00:00 change_column_types - 2026-07-07 18:01:15.802000+00:00 Pending: (none) If you don't specify a migrations file, the sqlite-utils migrate data.db command will scan the current directory and its subdirectories for files called migrations.py and apply any Migrations() instances it finds in them. You can also execute migrations from Python code using the migrations.apply(db) method, which is useful for building tools that manage their own database schemas over multiple versions. My own LLM tool has been using a version of this pattern for several years now, as shown in llm/embeddings_migrations.py . Prior art My favorite implementation of this pattern remains Django's Migrations , developed by Andrew Godwin based on his earlier project South . Fun fact: Andrew, Russ Keith-Magee, and I presented our competing approaches to schema migrations for Django on the Schema Evolution panel at the very first DjangoCon back in 2008! My attempt was called dmigrations , developed with a team at Global Radio in London. Django's migrations can be automatically generated from model definitions and include the ability to roll back to a previous version. The sqlite-utils approach is deliberately simpler: unlike Django, sqlite-utils encourages programmatic table creation rather than a model definition ORM, so there isn't anything we can use to automatically generate migrations. I decided to skip rollback, since in my experience it's a feature that is rarely used. With a SQLite project, an easy way to achieve rollback is to create a copy of your database file before you apply the migrations! Migrating from sqlite-migrate The design of sqlite-utils migrations is three years old now - I had originally released it as a separate package called sqlite-migrate , which never quite graduated beyond a beta release. I've used that package in enough places now that I'm confident in the design, so I've decided to promote it to a feature of sqlite-utils to make it available by default to all of the other tools in the growing sqlite-utils/Datasette/LLM ecosystem. I made one last release of sqlite-migrate , which switches it to depend on sqlite-utils>=4 and replaces the __init__.py file with the following: from sqlite_utils import Migrations __all__ = [ "Migrations" ] Any existing project that depends on sqlite-migrate should continue to work without alterations. Everything else in sqlite-utils 4.0 Here are the release notes for this version, with some inline annotations: The 4.0 release includes some minor backwards-incompatible fixes (hence the major version number bump) and introduces three major new features: Database migrations , providing a structured mechanism for evolving a project’s schema over time. ( #752 ) I think of migrations as the signature new feature, hence this blog post. Nested transaction support via db.atomic() , plus numerous improvements to how transactions work across the library. ( #755 ) sqlite-utils has long had a confused relationship with database transactions, partly because when I started designing the library back in 2018 I didn't yet have a great feel for how those worked in SQLite itself. Adding migrations to the core library made me determined to finally crack this nut, since transactions make migration systems a whole lot safer and easier to reason about. I ended up building this around a db.atomic() context manager which looks like this: with db . atomic (): db . table ( "dogs" ). insert ({ "id" : 1 , "name" : "Cleo" }, pk = "id" ) db . table ( "dogs" ). insert ({ "id" : 2 , "name" : "Pancakes" }) SQLite supports Savepoints , and as a result db.atomic() can be nested to carry out transactions inside of transactions. It's pretty neat! Support for compound foreign keys , including creation, transformation and introspection through table.foreign_keys . ( #594 ) This came about when I asked a coding agent to review all open issues and PRs for things that should be included in a 4.0 release since they would represent breaking changes if I added them later, and it correctly identified that compound foreign keys were exactly that kind of feature. I started with a breaking change to the table.foreign_keys introspection method, and then decided to see if Claude Fable 5 could handle the more fiddly job of integrating compound foreign key creation into the library. The API design it helped create felt exactly right to me - consistent with how the rest of the library worked already. Other notable changes include: Upserts now use SQLite’s INSERT ... ON CONFLICT ... DO UPDATE SET syntax, detect existing table primary keys automatically and reject records that are missing required primary key values. ( #652 ) This was the change that first pushed me to consider a breaking-change 4.0 version bump. I built this to help support sqlite-chronicle , which uses triggers to keep track of rows in a table that have been inserted, updated or deleted. db.query() now executes immediately and rejects statements that do not return rows; use db.execute() for writes and DDL. Probably the most disruptive breaking change - I've had to update a few places in my own code to switch from db.query() to db.execute() as a result. CSV and TSV imports now detect column types by default, while inserts into existing tables preserve those tables’ column types. ( #679 ) The sqlite-utils insert data.db creatures creatures.csv --detect-types flag was a later addition to allow column types (text, integer, real) to be automatically detected based on the data in a CSV. It should be the default, and releasing a 4.0 means I can make it so. table.extract() and extracts= no longer create lookup table records for all- null values. ( #186 ) The oldest issue addressed by this release - the underlying bug was opened (by me) in October 2020. See Upgrading from 3.x to 4.0 for details on backwards-incompatible changes. The detailed release notes for the features and fixes shipped during the 4.0 pre-release cycle are available in 4.0a0 , 4.0a1 , 4.0rc1 , 4.0rc2 , 4.0rc3 and 4.0rc4 . The upgrade guide was entirely written by Claude Fable 5, Claude Opus 4.8 and GPT-5.5. The same is true of the release notes. This is the kind of documentation I've slowly become comfortable outsourcing to the robots. It doesn't need to convince people of anything, or express any opinions - its job is to be as accurate and detailed as possible. I've reviewed the release notes closely and can confirm they are accurate and comprehensive. Claude Fable 5 helped a lot I released the first alpha of sqlite-utils 4.0 over a year ago . I've been dragging my heels on the stable release because of the amount of work it would take to track down and clean up the many other minor design flaws that a major version number allowed me to take on. Assistance from Claude Fable 5 (and to a lesser extent Opus 4.8 and GPT-5.5) gave me just the boost I needed to overcome inertia and make the most of the time I could afford to spend on this library. Fable has really good taste in API design, and is relentlessly proactive if you give it a more open goal. My most successful prompt was a review task that I issued against what I thought was the last release candidate: review the changes on main since the last tagged 3.x release - I am about to ship them as sqlite-utils 4.0, a stable version that promises no backwards-incompatible fixes for a very long time. review the changelog and upgrade guide, and write yourself scratch scripts to try out all of the new features in v4 - save those scripts but don't commit them I tried this with GPT-5.5 xhigh in Codex Desktop and Fable 5 in Claude Code. GPT-5.5 wrote 5 Python scripts and didn't turn up anything particularly interesting - its final report is here . Fable 5 wrote 12 scripts , identified 4 release blockers and 10 additional issues in its report , and built a neat combined repro script , which, when run, output the following: === 1. Failed db.execute() write leaves an implicit transaction open === in_transaction after failed write: True BUG: table 'other' silently lost when connection closed === 2. Leading ';' bypasses the query() first-token scanner === BUG: raised OperationalError: no such savepoint: sqlite_utils_query BUG: row persisted despite rollback (count=1) === 3. Rejected write PRAGMA via query() still takes effect === BUG: user_version=5 after 'rejected' statement (docs say no effect) === 4. Implicit compound FK resolves pk columns in table order, not PK order === BUG: other_columns reported as ('b', 'a'), should be ('a', 'b') BUG: transform of valid data raised IntegrityError: FOREIGN KEY constraint failed === 5. ForeignKey (now a dataclass) is no longer hashable === BUG: cannot use 'sqlite_utils.db.ForeignKey' as a set element (unhashable type: 'ForeignKey') === 6. Mixed ForeignKey objects and tuples in foreign_keys= rejected === BUG: foreign_keys= should be a list of tuples === 7. insert --csv into an EXISTING table transforms its column types === BUG: existing zip '01234' is now 1234 (column type: int) === 8. insert(pk=, alter=True) regression: InvalidColumns before alter runs === BUG: InvalidColumns: Invalid primary key column ['id'] for table t with columns ['a'] === 9. migrate --stop-before an already-applied migration applies everything === BUG: m2 was applied despite --stop-before m1 (m1 already applied) === 10. ensure_autocommit_on() silently commits an open transaction === BUG: row survived rollback (count=1) - transaction was committed I found myself agreeing with almost all of them. Here's the PR with 16 commits where we worked through them in turn. There's no doubt in my mind that sqlite-utils 4.0 is a significantly higher-quality release than if I had built it without the assistance of the latest frontier models. Tags: schema-migrations , projects , sqlite , ai , sqlite-utils , annotated-release-notes , generative-ai , llms , ai-assisted-programming , anthropic , claude , agentic-engineering , claude-mythos-fable

ReadSource

Simon Willison LLMs / 1:00 AM

sqlite-utils 4.0rc2, mostly written by Claude Fable (for about $149.25)

I wrote about the sqlite-utils 4.0rc1 release a couple of weeks ago. Since we only have Claude Fable on our Max subscriptions for a few more days, I decided to see if it could help me get to a 4.0 stable release that I felt truly comfortable about, since I try to keep to SemVer and like my incompatible major versions to be as rare as possible. I started with this prompt, in Claude Code for web on my iPhone: Final review before shipping a stable 4.0 release - very important to spot any last minute things that would be a breaking change if we fix them later Here's that initial report it created for me. There were some significant problems that I hadn't myself encountered yet - 5 that Fable categorized as "release blockers". Here's the worst of the bunch: 1. delete_where() never commits and poisons the connection (data loss) Table.delete_where() ( sqlite_utils/db.py:2948 ) runs its DELETE via a bare self.db.execute() with no atomic() wrapper — compare Table.delete() at db.py:2944 , which wraps correctly. The connection is left in_transaction=True , so every subsequent atomic() call takes the savepoint branch ( db.py:430-440 ) and never commits either. Reproduced end-to-end: db = sqlite_utils . Database ( "dw.db" ) db [ "t" ]. insert_all ([{ "id" : i } for i in range ( 3 )], pk = "id" ) db [ "t" ]. delete_where ( "id = ?" , [ 0 ]) # conn.in_transaction is now True db [ "t" ]. insert ({ "id" : 50 }) db [ "u" ]. insert ({ "a" : 1 }) db . close () # Reopen: rows are [0, 1, 2] — the delete, row 50, AND table u are all gone. That's a really bad bug! Very glad I didn't ship that, although at least it would have been a bug I could fix in a 4.0.1 point release, not a design flaw that would force a 5.0. Over the course of 37 prompts, 34 commits and +1,321 -190 code changes over 30 separate files, we worked through the entire set of feedback in turn, making several other design improvements along the way. A weird thing about coding agents is that harder tasks like this one actually provide more opportunity to do other things at the same time, since the agent sometimes needs 10-15 minutes to churn away on a new task. I went out to enjoy the Half Moon Bay 4th of July parade, occasionally checking in and prompting the next step for Fable from my phone. Full details in the PR and this shared transcript . I switched to my laptop for the final review, which I conducted through GitHub's PR interface. The most significant changes relate to transaction handling, which was the signature new feature in the earlier RC . The new RC now includes comprehensive documentation on the new transaction model, the intro to which I'll quote here in full: Every method in this library that writes to the database - insert() , upsert() , update() , delete() , delete_where() , transform() , create_table() , create_index() , enable_fts() and the rest - runs inside its own transaction and commits it before returning. Your changes are saved to disk as soon as the method call finishes: db = Database ( "data.db" ) db . table ( "news" ). insert ({ "headline" : "Dog wins award" }) # The new row is already saved - no commit() required The same applies to raw SQL executed with db.execute() - a write statement is committed as soon as it has run. You never need to call commit() , and you do not need to close the database to persist your changes. There are exactly two situations where you need to think about transactions: You want to group several write operations together, so they either all succeed or all fail - use db.atomic() . You are managing a transaction yourself with db.begin() , in which case nothing is committed until you commit - the library will never commit a transaction you opened. In reviewing Fable's documentation - I find that reviewing the documentation edits first is an excellent way to build an initial understanding of what has changed - I spotted this detail : db.atomic() and the automatic per-method transactions are designed for connections in Python's default transaction handling mode. Connections created with the Python 3.12+ sqlite3.connect(..., autocommit=True) or autocommit=False options are not supported, because commit() and rollback() behave differently on those connections. I admit I hadn't thought about how sqlite-utils would react to the more recent autocommit setting , added in Python 3.12. It turns out "behave differently on those connections" equated to almost the entire test suite failing, so I worked with the model to ensure that this difference would not break how the library works. And a final review by GPT-5.5 I used to think that the idea of having one model review the work of another was somewhat absurd - it felt weirdly superstitious. The problem is it really does work - I've started habitually having Anthropic's best model review OpenAI's work and vice versa, because I've had that turn up interesting results often enough to be valuable. I prompted Codex Desktop and GPT-5.5 xhigh with the following: Review changes since the last RC. Also confirm that the changelog is up-to-date. Which was enough to turn up two issues worth investigating: Findings [P1] sqlite_utils/db.py:663 db.query() now rejects non-row statements only after calling db.execute() , and sqlite_utils/db.py:705 auto-commits those writes first. So db.query("update ...") raises ValueError but the update is already committed. That is a surprising side effect for a method documented as “can only be used with SQL that returns rows.” [P1] sqlite_utils/db.py:672 INSERT ... RETURNING through db.query() only commits after the returned generator is fully exhausted. db.query("insert ... returning ...") without iteration, or common next(db.query(...)) usage, leaves the transaction open and the write can be rolled back on close. This contradicts docs/changelog.rst:15 and docs/python-api.rst:232 , which say it takes effect without iteration. I pasted that into a fresh Fable session, which ran some experiments to confirm the problem: Both findings were confirmed. db.query() called self.execute() first, which auto-commits writes, and only then checked cursor.description — so db.query("update ...") committed the update before raising ValueError . And the INSERT ... RETURNING commit lived at the end of the returned generator, so it never fired unless you exhausted the iterator — next(db.query(...)) or an un-iterated call left the transaction open, contradicting what the changelog and docs promise. Here's the PR with the fix, and the full Claude Code transcript . Reviewing this code helped me build a better mental model of the edge cases of SQLite transaction semantics! For an estimated (unsubsidized) cost of $149.25 I upgraded to the Claude Max $200/month plan (I was previously on $100/month) to increase my Fable allowance for the remaining time until the July 7th Fablepocalypse , when even Claude Max subscribers will have to pay full API cost for the model. I was curious as to how much this would have cost me if I had been paying those costs directly. At first I thought those numbers weren't available to me since I had run the work remotely using Claude Code for web, and then I realized I could run AgentsView inside that existing session to get that cost estimate! Run "uvx agentsview --help"​ and then use that tool to calculate the cost of this session Claude figured out how to use the session list --include-children command and came out with the following: Transcript Model Cost Main session claude-fable-5 $141.02 API-surface sweep agent claude-fable-5 $2.40 Transactions/atomic review agent claude-fable-5 $2.39 Post-rc1 commits review agent claude-fable-5 $1.72 Migrations review agent claude-fable-5 $1.40 Prompt-counting agent claude-opus-4-8 $0.32 Total $149.25 I'm very glad I'm on that subscription! I really should have followed my own advice and leaned more heavily into subagents with cheaper models. Here's what claude.ai/settings/usage is showing me right now: I have several other major Fable-driven projects on the go right now as well, with the goal of hitting 100% on that Fable bar just in time for the price increase. The full release notes for sqlite-utils 4.0rc2 Here are the full release notes for the RC. I had Fable add these to an "Unreleased" section of the changelog as each change landed, reviewing them as it went. This has the neat side effect that the commit history of the changelog acts as a concise summary of each of the changes that went into the release. In the past I've had a policy of writing release notes by hand, but honestly these are better than I would have created myself. Release notes are a great example of writing that I'm OK to outsource to agents because they need to be boring, predictable and accurate. Breaking changes: Write statements executed with db.execute() are now committed automatically, unless a transaction is already open in which case they join it. Previously they opened an implicit transaction that stayed open until something committed it - writes appeared to work when read on the same connection but were silently rolled back when the connection closed. Code that relied on rolling back uncommitted db.execute() writes should use the new db.begin() method to open an explicit transaction first. The transaction model is documented in full at Transactions and saving your changes . db.query() now executes its SQL as soon as it is called, rather than waiting until the returned generator is first iterated. Rows are still fetched lazily during iteration. SQL errors are now raised at the call site, statements such as INSERT ... RETURNING are executed and committed immediately without needing to iterate over their results, and passing a statement that returns no rows - previously a silent no-op - now raises a ValueError recommending db.execute() instead. A statement rejected this way is rolled back before the error is raised, so it has no effect on the database. Python API validation errors now raise ValueError instead of AssertionError . Previously invalid arguments - such as create_table() with no columns, transform() on a table that does not exist, or passing both ignore=True and replace=True - were rejected using bare assert statements, which are silently skipped when Python runs with the -O flag. Code that caught AssertionError for these cases should catch ValueError instead. table.upsert() and table.upsert_all() now raise PrimaryKeyRequired if a record is missing a value for any primary key column, or has a value of None for one. Previously such records - which can never match an existing row - were quietly inserted as brand new rows, or triggered a confusing KeyError after the insert had already taken place. db.enable_wal() and db.disable_wal() now raise a sqlite_utils.db.TransactionError if called while a transaction is open. Previously they would silently commit the open transaction as a side effect of changing the journal mode, breaking the rollback guarantee of db.atomic() and of user-managed transactions. The View class no longer has an enable_fts() method. It existed only to raise NotImplementedError , since full-text search is not supported for views - calling it now raises AttributeError instead, and the method no longer appears in the API reference. The sqlite-utils enable-fts command shows a clean error when pointed at a view. The no-op -d/--detect-types flag has been removed from the insert and upsert commands. Type detection has been the default for CSV/TSV data since 4.0a1, so the flag did nothing - invocations using it should simply drop it. --no-detect-types remains available to disable detection. Database() now raises a sqlite_utils.db.TransactionError if passed a connection created with the Python 3.12+ sqlite3.connect(..., autocommit=True) or autocommit=False options. commit() and rollback() behave differently on those connections, which previously caused every write made by the library to be silently discarded when the connection closed. Everything else: Fixed a bug where table.delete_where() , table.optimize() and table.rebuild_fts() did not commit their changes, leaving the connection inside an open transaction. Their work - and any subsequent writes - could then be silently rolled back when the connection was closed. All three now use db.atomic() , consistent with the other write methods. The sqlite-utils drop-table command now refuses to drop a view, and drop-view refuses to drop a table. Previously each would silently drop the wrong type of object if the name matched. Both now exit with an error suggesting the correct command to use. Migrations applied by the new migrations system now run inside a transaction, together with the record of the migration having been applied. If a migration raises an exception its changes are rolled back and it stays pending, so it can be safely re-applied after the error is fixed. Migrations that cannot run inside a transaction, such as those executing VACUUM , can opt out using @migrations(transactional=False) - see Migrations and transactions . table.upsert() and table.upsert_all() now detect the primary key or compound primary key of an existing table, so the pk= argument is no longer required when upserting into a table that already has a primary key. db.table(table_name).insert({}) can now be used to insert a row consisting entirely of default values into an existing table, using INSERT INTO ... DEFAULT VALUES . ( #759 ) Improvements to the sqlite-utils migrate command: --stop-before values that do not match any known migration are now an error instead of being silently ignored, --stop-before now works correctly with migration files that still use the older sqlite_migrate.Migrations class, and --list is now a read-only operation that no longer creates the database file or the migrations tracking table. migrations.applied() now returns migrations in the order they were applied. New db.begin() , db.commit() and db.rollback() methods for taking manual control of transactions, as an alternative to the db.atomic() context manager. New documentation: Transactions and saving your changes describes how transactions work and when changes are committed, and a new Upgrading page details the changes needed to move between major versions. Tags: projects , sqlite , ai , sqlite-utils , annotated-release-notes , generative-ai , llms , anthropic , claude , llm-pricing , coding-agents , claude-code , agentic-engineering , gpt , claude-mythos-fable

ReadSource

Simon Willison LLMs / 9:23 PM

What's new in Claude Sonnet 5

What's new in Claude Sonnet 5 Claude Sonnet 5 came out this morning . I always head straight for the "what's new" developer docs because they tend to have more actionable information than the official announcement post. Anthropic say of Sonnet 5 that "its performance is close to that of Opus 4.8, but at lower prices". The system card helps explain how they were able to release the model without being blocked by the US government: Sonnet 5 is significantly less capable at cyber tasks than Mythos 5: its safeguards are thus similar to those we apply to Opus 4.7 and Opus 4.8 (models that are more capable than Sonnet 5 but much less capable than Mythos 5). Of note from the "what's new" API changes: Sampling parameters temperature , top_p , top_k are no longer supported. It has a 1 million token context window and 128,000 maximum output tokens. It features "the same set of tools and platform features as Claude Sonnet 4.6" Adaptive thinking is on by default, unless you specify "thinking": {type: "disabled"} . The pricing is the same as Sonnet 4.6: $3/million input, $15/million input, with an introductory discount to $2/$10 until 31st August. But... The model has a new tokenizer, where "The same input text produces approximately 30% more tokens than on Claude Sonnet 4.6." - effectively a 30% price increase. I used my Claude Token Counter tool to try out the new tokenizer. Here are my results for several larger documents: Document Sonnet 4.6 Opus 4.7 Sonnet 5 Universal Declaration of Human Rights (English) 2,356 3,347 1.42x 3,341 1.42x Universal Declaration of Human Rights (Spanish) 3,572 4,753 1.33x 4,747 1.33x Universal Declaration of Human Rights (Chinese, Mandarin Simplified) 3,334 3,366 1.01x 3,360 1.01x sqlite_utils/db.py (4,279 lines of Python) 44,014 56,118 1.28x 56,113 1.27x So the new token is roughly 1.4x times more expensive for English, 1.33x for Spanish, 1.28x for Python code and effectively the same cost for Simplified Mandarin. Here's the pelican . It's nothing to write home about. Sonnet 5 thinks it looks like a goose. Via Hacker News Tags: ai , generative-ai , llms , anthropic , claude , llm-pricing , pelican-riding-a-bicycle , llm-release

ReadSource

GitHub Trending AI / 6:55 PM

invoke-ai/InvokeAI is trending in AI open source

invoke-ai/InvokeAI is a GitHub AI repository with 28,207 stars. Invoke is a leading creative engine for Stable Diffusion models, empowering professionals, artists, and enthusiasts to generate and create visual media using the latest AI-driven technologies. The solution offers an industry leading WebUI, and serves as the foundation for multiple commercial products.

ReadSource

Latest story in this edition: 5:21 PM

Back to front page