The AI Front Page

Topic Edition

Funding

30 stories from 12 sources across 1 topics.

Stories

30

Sources

12

Topics

1

For You lens

30 stories in this edition match your reader profile.

Reader signals

3

Searches

0

Matches

30

Top score

140

Tune For You

Edition Index

Topic, entity, and source map

Topics

Funding30 stories

Entities

Anthropic8 storiesOpenAI8 storiesClaude6 storiesGoogle6 storiesMicrosoft6 storiesCloudflare4 stories

Sources

TechCrunch AI6 storiesHacker News AI5 storiesArs Technica AI2 storiesAWS Machine Learning Blog2 storiesCloudflare AI Blog2 storiesLatent Space2 stories

The Decoder / 11:19 AM

Google CEO Pichai says Gemini's next leap depends on building "much larger base models"

Alphabet has raised its 2026 investment forecast to as much as $205 billion, saying demand continues to outpace spending. Google Cloud grew 82 percent in the second quarter. CEO Sundar Pichai says Google needs a larger base model for its next leap in AI and has kicked off an ambitious Gemini 4 training run. The article Google CEO Pichai says Gemini's next leap depends on building "much larger base models" appeared first on The Decoder .

ReadSource

VentureBeat AI / 7:02 PM

The agent security gap: 54% of enterprises have already had an AI agent incident, and most still let agents share credentials

Across 107 enterprises, AI agents are being given real access to systems and data while the controls meant to contain them lag behind. More than half have already had a confirmed agent security incident or a near-miss; only about a third give every agent its own scoped identity, and most agents still share credentials; and only three in ten isolate their highest-risk agents. The security stack is overwhelmingly borrowed from the model providers and hyperscalers rather than purpose-built for agents, spending remains a thin slice of the security budget, and enterprises are evenly split on whether their defenses are keeping pace with AI-enabled attackers. The result is an agent security gap — autonomous agents proliferating faster than the identity, isolation, and enforcement controls needed to hold them. This wave of VentureBeat Pulse Research examines how enterprises secure their AI agents: what tooling they run, how they manage agent identity and isolation, what has already gone wrong, how much they spend, and whether they believe their defenses are keeping pace with AI-enabled attackers. The central finding is an agent security gap — the distance between the autonomy enterprises are granting their agents and the controls in place to contain them. More than half of organizations (54%) have already experienced a confirmed agent security incident (18%) or a near-miss caught before harm (36%). The structural weakness beneath those numbers is identity: only about a third (32%) give every agent its own scoped, managed identity, while the rest report that some agents share credentials or that agents mostly run on shared API keys and human or service-account credentials. When agents share credentials, a single compromised or over-permissioned agent carries a wide blast radius — and only three in ten enterprises (30%) isolate their highest-risk agents in sandboxes to bound that radius. What makes the gap notable is how comfortable enterprises are inside it. The security stack is overwhelmingly provider-native — OpenAI’s guardrails (51%), Google’s and Microsoft’s cloud controls, and Anthropic’s managed-agent controls dominate, while the dedicated agent-security specialists barely register — and satisfaction with that borrowed stack is high, averaging 4.2 out of 5. Yet spending remains a thin slice of the security budget, only a third of enterprises believe their AI defenses are ahead of AI-enabled attackers, and a clear majority plan to change tooling within the year. Enterprises are satisfied with controls they are simultaneously preparing to replace. Methodology VentureBeat fielded this survey as part of its ongoing Pulse Research series, this instrument focused on enterprise agent security — the tooling, identity, isolation, and enforcement controls organizations use to secure autonomous AI agents. Responses are filtered to organizations with more than 100 employees (n=107; the survey’s smallest size band, 1–100 employees, is excluded), drawn from a single June 2026 wave. Because this is one wave rather than a pooled multi-month sample, the report reads cross-sectionally and does not infer month-over-month trends. Several questions were multiple-select, so those shares can sum to more than 100%. By role the sample is senior and buyer-credible: 45% are final decision-makers for AI purchases and another 30% recommenders or influencers. Managers (43%), individual contributors (24%), VPs and directors (15%), and the C-suite (11%) make up the seniority mix. By organization size the sample is mid-market-weighted: 251–1,000 (42%) and 101–250 (25%) employees lead, with 1,001–5,000 (19%), 5,001–10,000 (8%), and 10,001+ (7%) above them. Technology/Software is the largest industry at 23%, followed by Manufacturing (15%), Retail/E-commerce (14%), and Healthcare/Life Sciences (13%). At 107 respondents the sample is large enough to read directionally but should be treated as a directional signal rather than a precise measurement; it is self-selected and is not a probability sample. It skews toward the mid-market, so it is best read as the view from organizations actively standing up agent security rather than from the largest operators. Satisfaction ratings are computed on the respondents who answered each rating question; the overall satisfaction score reflects 82 of the 107 qualified respondents. Finding 1: The incidents are already here More than half have had an agent security incident or near-miss We asked whether organizations had experienced an agent security incident — a confirmed breach, or a near-miss caught before harm. Most that run agents in production had. This is the report’s defining number. More than half of organizations (54%) have already had an agent security event — 18% a confirmed incident and 36% a near-miss caught before it caused harm. Only 42% report nothing, and a small remainder either run no agents in production or don’t track such events. That so many report near-misses rather than only confirmed incidents is telling: enterprises are catching problems, but they are catching them close to the edge. The controls examined in the rest of this report — identity, isolation, enforcement — are what determine whether the next near-miss stays a near-miss. Exposure scales with company size, but containment does not. The incident-or-near-miss rate rises from 49% in the mid-market (companies with 101-1,000 employees) to 63% at larger enterprises (above 1,000 employees), while sandbox isolation of high-risk agents falls from 35% to 20%, and satisfaction with security tooling drops from 4.36 to 3.97. The organizations running the most agents across the most systems carry the most incidents and the least of the one control that bounds an incident's blast radius. Finding 2: The identity gap Only a third give every agent its own scoped identity We asked how enterprises manage the identity of their AI agents — whether each agent has its own credentials, or agents share them. Full per-agent identity is the exception. Rolled together, the overlapping answers show 69% of enterprises (74 of 107) with credential sharing somewhere in the agent fleet. Identity is the structural weakness beneath the incidents. Only about a third of enterprises (32%) give every agent its own scoped, managed identity — the precondition for least-privilege access and clean attribution. Nearly half (48%) say some agents have scoped identities but many still share credentials, and another 32% say agents mostly run on shared API keys or borrowed human and service-account credentials. (Respondents could describe more than one pattern across their agent fleet, so these overlap.) The consequence is direct: when agents share credentials, an over-permissioned or compromised agent can act with far more reach than intended, and forensics after an incident cannot cleanly tell which agent did what. The non-human identity problem — giving every agent its own governed identity — is the single largest unfinished piece of enterprise agent security. Moreover, a company’s agent credential posture is correlated with incidents. Organizations with credential sharing anywhere in the fleet were hit — with an incident or a near-miss in the past twelve months — at 63.5% (47 of 74). Organizations where every agent carries its own scoped identity were hit at 40.9% (9 of 22). The fully-scoped group is small, so for now the relationship is an association rather than proven causation, and the gap is concentrated in the mid-market — but within a single survey, a twenty-three point difference in incident rate suggests significance. Finding 3: Observe and enforce, but rarely isolate Only three in 10 sandbox their highest-risk agents We asked what an organization’s agent security posture looks like in practice — whether they observe, enforce, isolate, or some combination. The control that bounds damage is the least common. Monitoring and enforcement are reasonably common; containment is not. Roughly half of enterprises observe agent activity (47%) or enforce scoped permissions at runtime (49%), but only 30% isolate their highest-risk agents in sandboxes that bound the blast radius when the other controls fail. That ordering is backwards from a defense-in-depth standpoint: observation tells you what happened, enforcement tries to prevent it, but isolation is what limits the damage when prevention fails — and it is the control enterprises have adopted least. Combined with the identity gap in Finding 2, the picture is of agents that are watched and permissioned but rarely boxed in, which is precisely the configuration in which a single failure propagates. Finding 4: Security runs on borrowed, provider-native controls Guardrails from OpenAI, Google and Microsoft dominate; specialists barely register We asked which agent security tooling enterprises use, and which is their primary layer. The answer favors the model providers and hyperscalers over the dedicated security vendors. Enterprises are securing agents with tools that came bundled with their models and clouds. OpenAI’s guardrails lead at 51%, followed by Google’s and Microsoft’s cloud-native controls and Anthropic’s managed-agent controls — and when asked to name their single primary security layer, 82% name one of these provider-native offerings. The purpose-built agent-security category — Palo Alto’s Prisma AIRS, CrowdStrike, Cisco AI Defense, Zenity, HiddenLayer, Check Point’s Lakera, Okta for AI Agents, non-human identity platforms — barely registers, each in the low single digits, and only 5% run no dedicated tooling at all. As with retrieval and evaluation elsewhere in this series, the provider bundle is winning the default: enterprises reach first for the guardrails their platform ships, and the independent security layer that would address the identity and isolation gaps has not yet been adopted at scale. The provider-default pattern is consistent across both Q2 survey waves. In April–May (n=110), usage was led by the same names — OpenAI's controls at 26%, Azure at 15%, AWS at 14%, Google at 12% — with every dedicated agent-security specialist at 3% or below and one in ten using no dedicated tooling at all. The common finding from the two surveys: Enterprises are defaulting to the solutions provided by the platform they’re using, and the specialist category vendors have yet to become big players here. ( A note on reading these shares. As described in the methodology section, the respondent sample is self-selected and skews mid-market, and the usage question counted every vendor or approach a respondent has in place — so the figures measure presence in the security stack rather than spending or exclusivity. Individual vendor percentages therefore carry all the usual sample caveats. The structural pattern, however, held across both Q2 waves on two differently worded questions: provider-native and hyperscaler controls lead, and dedicated agent-security specialists remain in low single digits. Read the individual shares loosely and the pattern with confidence.) Finding 5: And enterprises are comfortable with it Satisfaction is high, even as incidents mount and identity lags We asked how satisfied enterprises are with their current agent security tooling. The comfort is notably out of step with the exposure documented above. Satisfaction with agent security tooling is high — 4.2 out of 5 overall, and 4.1 for value for money — among the most positive readings in this series. That is the striking part: enterprises are highly satisfied with a stack that is mostly borrowed provider guardrails, even though more than half have already had an incident or near-miss and only a third give their agents scoped identities. The comfort appears to rest on the convenience and low friction of provider-native controls rather than on demonstrated containment. It is a false comfort in the making — the same enterprises expressing satisfaction are, as Finding 8 shows, a clear majority planning to change tooling within the year, which suggests the confidence is thinner than the score implies. Finding 6: Budgets haven’t caught up Most spend under a tenth of the security budget on agents We asked what share of the security budget enterprises allocate to securing AI agents. For a fast-emerging risk, the allocation is modest. Spending on agent security is still a thin slice. The most common allocation is 6–10% of the security budget (46%), and a third of enterprises (34%) spend 5% or less; only a quarter (24%) devote more than a tenth. Given the incident rate in Finding 1 and the identity and isolation gaps in Findings 2 and 3, the budget looks like a lagging indicator — the risk has arrived faster than the funding to address it. The enterprises spending more than a tenth of their security budget on agents are a distinct minority, and they are likely the ones building the scoped-identity and isolation controls the rest have not. Finding 7: The arms race is even, at best Only a third think their AI defenses are ahead of AI-enabled attackers We asked how enterprises assess the balance between their AI-enabled defenses and AI-enabled attackers. Confidence is far from settled. Enterprises are split on whether they are winning. Only about a third (35%) believe their AI-enabled defenses are ahead of AI-enabled attackers; the rest are less sure — 32% call it roughly even, 21% think attackers are ahead, and another 21% say it is too early to tell. Taken together, a clear majority (53%) rate the balance as even or tilted toward the attacker. That uncertainty sits uneasily beside the high satisfaction of Finding 5: enterprises are content with their tooling yet unconvinced it is winning the contest it exists to win. In a domain where the offense is also compounding with AI, an even race is not a comfortable place to be. Finding 8: A security reshuffle is coming Nearly six in 10 plan to adopt or switch tooling within a year We asked whether enterprises plan to adopt a new, additional, or replacement agent security solution, and which they are considering. Few intend to stand pat. The security stack is not settled. While 41% have no plans to change, a clear majority (59%) intend to adopt a new, additional, or replacement agent security solution within twelve months, and 29% within the next quarter — a strong signal that, high satisfaction notwithstanding, enterprises know the current stack is provisional. Incidents are what start the buying cycle. Among organizations that have been hit, 42.1% plan to adopt, add, or replace agent security tooling within the next ninety days, against 14.0% of organizations with no incident — and after a confirmed incident it becomes majority behavior, at 52.6%. Getting hit also changes the threat assessment: 33.3% of hit organizations say AI-armed attackers are ahead of their defenses, against 8.0% of the unhit. Experience, in this data, is the strongest predictor of both urgency and pessimism. The consideration set still leans provider-native (OpenAI 34%, Google 30%, Anthropic 29%, Azure 25%), but the dedicated security vendors — Cloudflare, Cisco, Palo Alto, Okta, Check Point’s Lakera — draw early interest in the mid-to-high single digits, more than their current footprint. What the shopping does not yet include is the identity layer specifically. Twelve percent of the respondents include an agent-identity product — Okta for AI Agents, Microsoft Entra Agent ID, or a non-human identity platform — anywhere in their consideration set, and among the credential-sharing organizations that have already had an incident, identity consideration is essentially unchanged, at roughly one in ten. The control most directly implicated by the incident data is the one largely missing from the purchase plans. Whether this wave hardens the provider-native default or finally opens the door to purpose-built agent security — the identity and isolation controls the incidents call for — is the question this series will keep tracking. The bottom line: A security gap that autonomy will test first Organizations with more than 100 employees are giving AI agents real reach into systems and data while securing them with controls built for something else. More than half have already had an incident or near-miss; only a third give every agent its own scoped identity, and most still share credentials; only three in ten isolate their highest-risk agents; and the stack doing this work is overwhelmingly borrowed from the model providers and hyperscalers rather than purpose-built for agents. The uncomfortable pairing is confidence with exposure: satisfaction with the current tooling is among the highest in this series, yet spending is a thin slice of the security budget, only a third believe their defenses are ahead of AI-enabled attackers, and a clear majority are already planning to replace what they have. At 107 respondents in a single wave this is a directional read, skewed toward the mid-market — but the direction is clear: agent adoption is running ahead of agent security, and the controls that matter most when something fails — scoped identity and isolation — are the ones enterprises have built least. The agent security gap is not a coverage problem that a provider guardrail will close on its own; it is a problem of identity, isolation, and enforcement built for autonomous software. The open question for later waves is whether enterprises close it deliberately — or whether a confirmed incident closes it for them. Based on survey responses from 107 qualified enterprise respondents (100+ employees), drawn from a single June 2026 wave. This is a directional read, not a precise measurement — the sample is self-selected and skews mid-market, so it's best read as the view from organizations actively standing up agent security rather than from the largest operators. Respondents are senior and buyer-credible (45% final decision-makers, 30% recommenders/influencers), spanning managers through the C-suite, and drawn primarily from Technology/Software, Manufacturing, Retail/E-commerce, and Healthcare/Life Sciences.

ReadSource

Simon Willison LLMs / 7:32 PM

sqlite-utils 4.0, now with database schema migrations

This morning I released sqlite-utils 4.0 , the 124th release of that project and the first major version bump since 3.0 in November 2020. In addition to some small but significant breaking changes (described in this upgrade guide ), this version introduces three major features: database migrations , nested transactions (via a new db.atomic() method), and support for compound foreign keys . Database schema migrations using sqlite-utils Schema migrations define a sequence of changes to be made to a SQLite database, plus a mechanism for tracking which migrations have been applied and applying any that are found to be pending. Migrations are defined in Python files using the sqlite-utils Python library , which includes a powerful table.transform() method providing enhanced alter table capabilities that are not supported by SQLite's ALTER TABLE statement. ( table.transform() implements the pattern recommended by the SQLite documentation - create a new temporary table with the new schema, copy across the data, then drop the old table and rename the temporary one in its place.) Here's an example migration file which creates a table called creatures , adds an additional column to it in a second step, then changes the types of two of the columns in a third: from sqlite_utils import Migrations migrations = Migrations ( "creatures" ) @ migrations () def create_table ( db ): db [ "creatures" ]. create ( { "id" : int , "name" : str , "species" : str }, pk = "id" , ) @ migrations () def add_weight ( db ): db [ "creatures" ]. add_column ( "weight" , float ) @ migrations () def change_column_types ( db ): db [ "creatures" ]. transform ( types = { "species" : int , "weight" : str }) Save that as migrations.py and run it against a fresh database like this: uvx sqlite-utils migrate data.db migrations.py Then if you check the schema of that database: uvx sqlite-utils schema data.db You'll see this SQL: CREATE TABLE " _sqlite_migrations " ( " id " INTEGER PRIMARY KEY , " migration_set " TEXT , " name " TEXT , " applied_at " TEXT ); CREATE UNIQUE INDEX " idx__sqlite_migrations_migration_set_name " ON " _sqlite_migrations " ( " migration_set " , " name " ); CREATE TABLE " creatures " ( " id " INTEGER PRIMARY KEY , " name " TEXT , " species " INTEGER , " weight " TEXT ); The _sqlite_migrations table is used to keep track of which migration functions have been run. The creatures table above is the schema after all three migrations have been applied. To see a list of migrations, both pending and applied, run this: uvx sqlite-utils migrate data.db migrations.py --list Output: Migrations for: creatures Applied: create_table - 2026-07-07 17:58:41.360051+00:00 add_weight - 2026-07-07 17:58:41.360608+00:00 change_column_types - 2026-07-07 18:01:15.802000+00:00 Pending: (none) If you don't specify a migrations file, the sqlite-utils migrate data.db command will scan the current directory and its subdirectories for files called migrations.py and apply any Migrations() instances it finds in them. You can also execute migrations from Python code using the migrations.apply(db) method, which is useful for building tools that manage their own database schemas over multiple versions. My own LLM tool has been using a version of this pattern for several years now, as shown in llm/embeddings_migrations.py . Prior art My favorite implementation of this pattern remains Django's Migrations , developed by Andrew Godwin based on his earlier project South . Fun fact: Andrew, Russ Keith-Magee, and I presented our competing approaches to schema migrations for Django on the Schema Evolution panel at the very first DjangoCon back in 2008! My attempt was called dmigrations , developed with a team at Global Radio in London. Django's migrations can be automatically generated from model definitions and include the ability to roll back to a previous version. The sqlite-utils approach is deliberately simpler: unlike Django, sqlite-utils encourages programmatic table creation rather than a model definition ORM, so there isn't anything we can use to automatically generate migrations. I decided to skip rollback, since in my experience it's a feature that is rarely used. With a SQLite project, an easy way to achieve rollback is to create a copy of your database file before you apply the migrations! Migrating from sqlite-migrate The design of sqlite-utils migrations is three years old now - I had originally released it as a separate package called sqlite-migrate , which never quite graduated beyond a beta release. I've used that package in enough places now that I'm confident in the design, so I've decided to promote it to a feature of sqlite-utils to make it available by default to all of the other tools in the growing sqlite-utils/Datasette/LLM ecosystem. I made one last release of sqlite-migrate , which switches it to depend on sqlite-utils>=4 and replaces the __init__.py file with the following: from sqlite_utils import Migrations __all__ = [ "Migrations" ] Any existing project that depends on sqlite-migrate should continue to work without alterations. Everything else in sqlite-utils 4.0 Here are the release notes for this version, with some inline annotations: The 4.0 release includes some minor backwards-incompatible fixes (hence the major version number bump) and introduces three major new features: Database migrations , providing a structured mechanism for evolving a project’s schema over time. ( #752 ) I think of migrations as the signature new feature, hence this blog post. Nested transaction support via db.atomic() , plus numerous improvements to how transactions work across the library. ( #755 ) sqlite-utils has long had a confused relationship with database transactions, partly because when I started designing the library back in 2018 I didn't yet have a great feel for how those worked in SQLite itself. Adding migrations to the core library made me determined to finally crack this nut, since transactions make migration systems a whole lot safer and easier to reason about. I ended up building this around a db.atomic() context manager which looks like this: with db . atomic (): db . table ( "dogs" ). insert ({ "id" : 1 , "name" : "Cleo" }, pk = "id" ) db . table ( "dogs" ). insert ({ "id" : 2 , "name" : "Pancakes" }) SQLite supports Savepoints , and as a result db.atomic() can be nested to carry out transactions inside of transactions. It's pretty neat! Support for compound foreign keys , including creation, transformation and introspection through table.foreign_keys . ( #594 ) This came about when I asked a coding agent to review all open issues and PRs for things that should be included in a 4.0 release since they would represent breaking changes if I added them later, and it correctly identified that compound foreign keys were exactly that kind of feature. I started with a breaking change to the table.foreign_keys introspection method, and then decided to see if Claude Fable 5 could handle the more fiddly job of integrating compound foreign key creation into the library. The API design it helped create felt exactly right to me - consistent with how the rest of the library worked already. Other notable changes include: Upserts now use SQLite’s INSERT ... ON CONFLICT ... DO UPDATE SET syntax, detect existing table primary keys automatically and reject records that are missing required primary key values. ( #652 ) This was the change that first pushed me to consider a breaking-change 4.0 version bump. I built this to help support sqlite-chronicle , which uses triggers to keep track of rows in a table that have been inserted, updated or deleted. db.query() now executes immediately and rejects statements that do not return rows; use db.execute() for writes and DDL. Probably the most disruptive breaking change - I've had to update a few places in my own code to switch from db.query() to db.execute() as a result. CSV and TSV imports now detect column types by default, while inserts into existing tables preserve those tables’ column types. ( #679 ) The sqlite-utils insert data.db creatures creatures.csv --detect-types flag was a later addition to allow column types (text, integer, real) to be automatically detected based on the data in a CSV. It should be the default, and releasing a 4.0 means I can make it so. table.extract() and extracts= no longer create lookup table records for all- null values. ( #186 ) The oldest issue addressed by this release - the underlying bug was opened (by me) in October 2020. See Upgrading from 3.x to 4.0 for details on backwards-incompatible changes. The detailed release notes for the features and fixes shipped during the 4.0 pre-release cycle are available in 4.0a0 , 4.0a1 , 4.0rc1 , 4.0rc2 , 4.0rc3 and 4.0rc4 . The upgrade guide was entirely written by Claude Fable 5, Claude Opus 4.8 and GPT-5.5. The same is true of the release notes. This is the kind of documentation I've slowly become comfortable outsourcing to the robots. It doesn't need to convince people of anything, or express any opinions - its job is to be as accurate and detailed as possible. I've reviewed the release notes closely and can confirm they are accurate and comprehensive. Claude Fable 5 helped a lot I released the first alpha of sqlite-utils 4.0 over a year ago . I've been dragging my heels on the stable release because of the amount of work it would take to track down and clean up the many other minor design flaws that a major version number allowed me to take on. Assistance from Claude Fable 5 (and to a lesser extent Opus 4.8 and GPT-5.5) gave me just the boost I needed to overcome inertia and make the most of the time I could afford to spend on this library. Fable has really good taste in API design, and is relentlessly proactive if you give it a more open goal. My most successful prompt was a review task that I issued against what I thought was the last release candidate: review the changes on main since the last tagged 3.x release - I am about to ship them as sqlite-utils 4.0, a stable version that promises no backwards-incompatible fixes for a very long time. review the changelog and upgrade guide, and write yourself scratch scripts to try out all of the new features in v4 - save those scripts but don't commit them I tried this with GPT-5.5 xhigh in Codex Desktop and Fable 5 in Claude Code. GPT-5.5 wrote 5 Python scripts and didn't turn up anything particularly interesting - its final report is here . Fable 5 wrote 12 scripts , identified 4 release blockers and 10 additional issues in its report , and built a neat combined repro script , which, when run, output the following: === 1. Failed db.execute() write leaves an implicit transaction open === in_transaction after failed write: True BUG: table 'other' silently lost when connection closed === 2. Leading ';' bypasses the query() first-token scanner === BUG: raised OperationalError: no such savepoint: sqlite_utils_query BUG: row persisted despite rollback (count=1) === 3. Rejected write PRAGMA via query() still takes effect === BUG: user_version=5 after 'rejected' statement (docs say no effect) === 4. Implicit compound FK resolves pk columns in table order, not PK order === BUG: other_columns reported as ('b', 'a'), should be ('a', 'b') BUG: transform of valid data raised IntegrityError: FOREIGN KEY constraint failed === 5. ForeignKey (now a dataclass) is no longer hashable === BUG: cannot use 'sqlite_utils.db.ForeignKey' as a set element (unhashable type: 'ForeignKey') === 6. Mixed ForeignKey objects and tuples in foreign_keys= rejected === BUG: foreign_keys= should be a list of tuples === 7. insert --csv into an EXISTING table transforms its column types === BUG: existing zip '01234' is now 1234 (column type: int) === 8. insert(pk=, alter=True) regression: InvalidColumns before alter runs === BUG: InvalidColumns: Invalid primary key column ['id'] for table t with columns ['a'] === 9. migrate --stop-before an already-applied migration applies everything === BUG: m2 was applied despite --stop-before m1 (m1 already applied) === 10. ensure_autocommit_on() silently commits an open transaction === BUG: row survived rollback (count=1) - transaction was committed I found myself agreeing with almost all of them. Here's the PR with 16 commits where we worked through them in turn. There's no doubt in my mind that sqlite-utils 4.0 is a significantly higher-quality release than if I had built it without the assistance of the latest frontier models. Tags: schema-migrations , projects , sqlite , ai , sqlite-utils , annotated-release-notes , generative-ai , llms , ai-assisted-programming , anthropic , claude , agentic-engineering , claude-mythos-fable

ReadSource

Simon Willison LLMs / 1:00 AM

sqlite-utils 4.0rc2, mostly written by Claude Fable (for about $149.25)

I wrote about the sqlite-utils 4.0rc1 release a couple of weeks ago. Since we only have Claude Fable on our Max subscriptions for a few more days, I decided to see if it could help me get to a 4.0 stable release that I felt truly comfortable about, since I try to keep to SemVer and like my incompatible major versions to be as rare as possible. I started with this prompt, in Claude Code for web on my iPhone: Final review before shipping a stable 4.0 release - very important to spot any last minute things that would be a breaking change if we fix them later Here's that initial report it created for me. There were some significant problems that I hadn't myself encountered yet - 5 that Fable categorized as "release blockers". Here's the worst of the bunch: 1. delete_where() never commits and poisons the connection (data loss) Table.delete_where() ( sqlite_utils/db.py:2948 ) runs its DELETE via a bare self.db.execute() with no atomic() wrapper — compare Table.delete() at db.py:2944 , which wraps correctly. The connection is left in_transaction=True , so every subsequent atomic() call takes the savepoint branch ( db.py:430-440 ) and never commits either. Reproduced end-to-end: db = sqlite_utils . Database ( "dw.db" ) db [ "t" ]. insert_all ([{ "id" : i } for i in range ( 3 )], pk = "id" ) db [ "t" ]. delete_where ( "id = ?" , [ 0 ]) # conn.in_transaction is now True db [ "t" ]. insert ({ "id" : 50 }) db [ "u" ]. insert ({ "a" : 1 }) db . close () # Reopen: rows are [0, 1, 2] — the delete, row 50, AND table u are all gone. That's a really bad bug! Very glad I didn't ship that, although at least it would have been a bug I could fix in a 4.0.1 point release, not a design flaw that would force a 5.0. Over the course of 37 prompts, 34 commits and +1,321 -190 code changes over 30 separate files, we worked through the entire set of feedback in turn, making several other design improvements along the way. A weird thing about coding agents is that harder tasks like this one actually provide more opportunity to do other things at the same time, since the agent sometimes needs 10-15 minutes to churn away on a new task. I went out to enjoy the Half Moon Bay 4th of July parade, occasionally checking in and prompting the next step for Fable from my phone. Full details in the PR and this shared transcript . I switched to my laptop for the final review, which I conducted through GitHub's PR interface. The most significant changes relate to transaction handling, which was the signature new feature in the earlier RC . The new RC now includes comprehensive documentation on the new transaction model, the intro to which I'll quote here in full: Every method in this library that writes to the database - insert() , upsert() , update() , delete() , delete_where() , transform() , create_table() , create_index() , enable_fts() and the rest - runs inside its own transaction and commits it before returning. Your changes are saved to disk as soon as the method call finishes: db = Database ( "data.db" ) db . table ( "news" ). insert ({ "headline" : "Dog wins award" }) # The new row is already saved - no commit() required The same applies to raw SQL executed with db.execute() - a write statement is committed as soon as it has run. You never need to call commit() , and you do not need to close the database to persist your changes. There are exactly two situations where you need to think about transactions: You want to group several write operations together, so they either all succeed or all fail - use db.atomic() . You are managing a transaction yourself with db.begin() , in which case nothing is committed until you commit - the library will never commit a transaction you opened. In reviewing Fable's documentation - I find that reviewing the documentation edits first is an excellent way to build an initial understanding of what has changed - I spotted this detail : db.atomic() and the automatic per-method transactions are designed for connections in Python's default transaction handling mode. Connections created with the Python 3.12+ sqlite3.connect(..., autocommit=True) or autocommit=False options are not supported, because commit() and rollback() behave differently on those connections. I admit I hadn't thought about how sqlite-utils would react to the more recent autocommit setting , added in Python 3.12. It turns out "behave differently on those connections" equated to almost the entire test suite failing, so I worked with the model to ensure that this difference would not break how the library works. And a final review by GPT-5.5 I used to think that the idea of having one model review the work of another was somewhat absurd - it felt weirdly superstitious. The problem is it really does work - I've started habitually having Anthropic's best model review OpenAI's work and vice versa, because I've had that turn up interesting results often enough to be valuable. I prompted Codex Desktop and GPT-5.5 xhigh with the following: Review changes since the last RC. Also confirm that the changelog is up-to-date. Which was enough to turn up two issues worth investigating: Findings [P1] sqlite_utils/db.py:663 db.query() now rejects non-row statements only after calling db.execute() , and sqlite_utils/db.py:705 auto-commits those writes first. So db.query("update ...") raises ValueError but the update is already committed. That is a surprising side effect for a method documented as “can only be used with SQL that returns rows.” [P1] sqlite_utils/db.py:672 INSERT ... RETURNING through db.query() only commits after the returned generator is fully exhausted. db.query("insert ... returning ...") without iteration, or common next(db.query(...)) usage, leaves the transaction open and the write can be rolled back on close. This contradicts docs/changelog.rst:15 and docs/python-api.rst:232 , which say it takes effect without iteration. I pasted that into a fresh Fable session, which ran some experiments to confirm the problem: Both findings were confirmed. db.query() called self.execute() first, which auto-commits writes, and only then checked cursor.description — so db.query("update ...") committed the update before raising ValueError . And the INSERT ... RETURNING commit lived at the end of the returned generator, so it never fired unless you exhausted the iterator — next(db.query(...)) or an un-iterated call left the transaction open, contradicting what the changelog and docs promise. Here's the PR with the fix, and the full Claude Code transcript . Reviewing this code helped me build a better mental model of the edge cases of SQLite transaction semantics! For an estimated (unsubsidized) cost of $149.25 I upgraded to the Claude Max $200/month plan (I was previously on $100/month) to increase my Fable allowance for the remaining time until the July 7th Fablepocalypse , when even Claude Max subscribers will have to pay full API cost for the model. I was curious as to how much this would have cost me if I had been paying those costs directly. At first I thought those numbers weren't available to me since I had run the work remotely using Claude Code for web, and then I realized I could run AgentsView inside that existing session to get that cost estimate! Run "uvx agentsview --help"​ and then use that tool to calculate the cost of this session Claude figured out how to use the session list --include-children command and came out with the following: Transcript Model Cost Main session claude-fable-5 $141.02 API-surface sweep agent claude-fable-5 $2.40 Transactions/atomic review agent claude-fable-5 $2.39 Post-rc1 commits review agent claude-fable-5 $1.72 Migrations review agent claude-fable-5 $1.40 Prompt-counting agent claude-opus-4-8 $0.32 Total $149.25 I'm very glad I'm on that subscription! I really should have followed my own advice and leaned more heavily into subagents with cheaper models. Here's what claude.ai/settings/usage is showing me right now: I have several other major Fable-driven projects on the go right now as well, with the goal of hitting 100% on that Fable bar just in time for the price increase. The full release notes for sqlite-utils 4.0rc2 Here are the full release notes for the RC. I had Fable add these to an "Unreleased" section of the changelog as each change landed, reviewing them as it went. This has the neat side effect that the commit history of the changelog acts as a concise summary of each of the changes that went into the release. In the past I've had a policy of writing release notes by hand, but honestly these are better than I would have created myself. Release notes are a great example of writing that I'm OK to outsource to agents because they need to be boring, predictable and accurate. Breaking changes: Write statements executed with db.execute() are now committed automatically, unless a transaction is already open in which case they join it. Previously they opened an implicit transaction that stayed open until something committed it - writes appeared to work when read on the same connection but were silently rolled back when the connection closed. Code that relied on rolling back uncommitted db.execute() writes should use the new db.begin() method to open an explicit transaction first. The transaction model is documented in full at Transactions and saving your changes . db.query() now executes its SQL as soon as it is called, rather than waiting until the returned generator is first iterated. Rows are still fetched lazily during iteration. SQL errors are now raised at the call site, statements such as INSERT ... RETURNING are executed and committed immediately without needing to iterate over their results, and passing a statement that returns no rows - previously a silent no-op - now raises a ValueError recommending db.execute() instead. A statement rejected this way is rolled back before the error is raised, so it has no effect on the database. Python API validation errors now raise ValueError instead of AssertionError . Previously invalid arguments - such as create_table() with no columns, transform() on a table that does not exist, or passing both ignore=True and replace=True - were rejected using bare assert statements, which are silently skipped when Python runs with the -O flag. Code that caught AssertionError for these cases should catch ValueError instead. table.upsert() and table.upsert_all() now raise PrimaryKeyRequired if a record is missing a value for any primary key column, or has a value of None for one. Previously such records - which can never match an existing row - were quietly inserted as brand new rows, or triggered a confusing KeyError after the insert had already taken place. db.enable_wal() and db.disable_wal() now raise a sqlite_utils.db.TransactionError if called while a transaction is open. Previously they would silently commit the open transaction as a side effect of changing the journal mode, breaking the rollback guarantee of db.atomic() and of user-managed transactions. The View class no longer has an enable_fts() method. It existed only to raise NotImplementedError , since full-text search is not supported for views - calling it now raises AttributeError instead, and the method no longer appears in the API reference. The sqlite-utils enable-fts command shows a clean error when pointed at a view. The no-op -d/--detect-types flag has been removed from the insert and upsert commands. Type detection has been the default for CSV/TSV data since 4.0a1, so the flag did nothing - invocations using it should simply drop it. --no-detect-types remains available to disable detection. Database() now raises a sqlite_utils.db.TransactionError if passed a connection created with the Python 3.12+ sqlite3.connect(..., autocommit=True) or autocommit=False options. commit() and rollback() behave differently on those connections, which previously caused every write made by the library to be silently discarded when the connection closed. Everything else: Fixed a bug where table.delete_where() , table.optimize() and table.rebuild_fts() did not commit their changes, leaving the connection inside an open transaction. Their work - and any subsequent writes - could then be silently rolled back when the connection was closed. All three now use db.atomic() , consistent with the other write methods. The sqlite-utils drop-table command now refuses to drop a view, and drop-view refuses to drop a table. Previously each would silently drop the wrong type of object if the name matched. Both now exit with an error suggesting the correct command to use. Migrations applied by the new migrations system now run inside a transaction, together with the record of the migration having been applied. If a migration raises an exception its changes are rolled back and it stays pending, so it can be safely re-applied after the error is fixed. Migrations that cannot run inside a transaction, such as those executing VACUUM , can opt out using @migrations(transactional=False) - see Migrations and transactions . table.upsert() and table.upsert_all() now detect the primary key or compound primary key of an existing table, so the pk= argument is no longer required when upserting into a table that already has a primary key. db.table(table_name).insert({}) can now be used to insert a row consisting entirely of default values into an existing table, using INSERT INTO ... DEFAULT VALUES . ( #759 ) Improvements to the sqlite-utils migrate command: --stop-before values that do not match any known migration are now an error instead of being silently ignored, --stop-before now works correctly with migration files that still use the older sqlite_migrate.Migrations class, and --list is now a read-only operation that no longer creates the database file or the migrations tracking table. migrations.applied() now returns migrations in the order they were applied. New db.begin() , db.commit() and db.rollback() methods for taking manual control of transactions, as an alternative to the db.atomic() context manager. New documentation: Transactions and saving your changes describes how transactions work and when changes are committed, and a new Upgrading page details the changes needed to move between major versions. Tags: projects , sqlite , ai , sqlite-utils , annotated-release-notes , generative-ai , llms , anthropic , claude , llm-pricing , coding-agents , claude-code , agentic-engineering , gpt , claude-mythos-fable

ReadSource

The Verge AI / 10:23 AM

OpenAI floats giving Trump administration 5 percent cut of AI boom

OpenAI has floated giving the US government a 5 percent ownership stake as a way of easing tensions with the Trump administration and blunting mounting public backlash against AI, according to the Financial Times. CEO Sam Altman argued that giving the public a financial interest in the company would be the best way to share […]

ReadSource

Latest story in this edition: 1:25 PM

Back to front page