Submitted:
19 October 2026
Posted:
20 October 2026
You are already at the latest version
Abstract
Context. Vibe coding, building software by describing it to AI systems and accepting the code they write, now produces products that people pay for. Evidence comes mostly from interviews and online posts, rarely from the records of a live system built by a non-programmer. Objective. To describe how one such product was built and kept under control over its first 104 days, starting the day before public launch. Method. A single-case study of Curie, a research platform built by an environmental engineer who does not write code, using a commercial AI app builder and external AI agents. I analysed all 2,334 commits, 1,940 instructions to the builder, 2,457 checkpoints and weekly code snapshots. Results. AI tools generated every code change: the builder made 972 commits (41.6%) and external agents 1,296 (55.5%). Agent commits were smaller (median 21 against 87 lines; rank-biserial r = 0.40, p < 0.001); durability did not differ consistently once timing was controlled. Half of all file edits revisited a file changed within 24 hours. The codebase grew 6.2-fold while decision points per thousand lines rose 94%. Tests first appeared on day 88, and a rollback on day 102 discarded 302 commits, including half the tests. The product reached about 500 paying users. Conclusions. Ten guardrails, four prompted directly by platform behaviour, form a ladder from prototype to team: three observed levels and a proposed fourth. With AI writing all the code, the founder's work was directing the AI and checking its output.

Keywords:
vibe coding
; AI-assisted software development
; AI coding agents
; end-user programming
; software evolution
; mining software repositories
1. Introduction
On 2 February 2025 Andrej Karpathy described a way of programming he called “vibe coding”: telling a large language model what you want, accepting the changes it proposes and seldom reading the code [1]. The phrase spread quickly because it named something many people were already doing, and before the end of the year Collins Dictionary had made it its word of the year [2]. Hosted AI app builders now let people with no programming background describe an application in plain language and publish it the same day, with a database, user accounts and payments included.
Most of what is known about vibe coding comes from what people say about it. Reviews of practitioner accounts describe a loop of generating, evaluating and revising code, report short-term gains in speed, and find little evidence about maintainability or long-term quality [3,4]. Interview, video and survey studies describe developers who regenerate output until something works, adjust their trust in the AI as they go, and differ sharply in their ability to verify what it produces [5,6,7,8]. Experienced engineers rarely hand over control at all; they keep design decisions and steer the agents they use [9]. Controlled experiments with AI coding assistants point in different directions, from a 55.8% speed-up on a bounded task [10] to a 19% slowdown for experienced developers working in codebases they knew well [11].
Three gaps remain. First, few studies follow a vibe-coded system from launch through months of operation with paying users. Second, the people studied are usually developers. Founders who cannot read code, and must therefore judge an AI’s work only by what the software does, are rarely observed directly, although a survey of vibe coders suggests that people without coding experience are the least able to verify what the AI produces [8]. Third, most evidence is self-reported. A product’s version history records every change, who or what made it and whether it lasted, and so offers a more objective view of how the work was done, provided its known pitfalls are respected [12,13].
This paper addresses those gaps with a single-case study of Curie, a research platform that I built with a commercial AI app builder and external AI agents, and that reached about 500 paying users within three months of opening to the public. I combine four records that together describe both the code and the person directing it: the full commit history, the conversation log with the builder AI, the platform’s checkpoint and deployment records, and weekly snapshots of the codebase. I ask four questions:
- RQ1. How was the work divided between the person directing it, the platform’s builder AI and external AI agents?
- RQ2. How did the work and the codebase change over time, and how stable was the code that was written?
- RQ3. Which platform-level problems arose while the product was in production?
- RQ4. Which practices did a non-programmer adopt to stay in control, and how can they be organised for products that grow?
The paper makes five contributions. It gives a repository-level account of a vibe-coded product built by a non-programmer and operated with paying users. It quantifies the human side of vibe coding: how many instructions were needed, how often versions were deployed and how much active time the work took. It compares two kinds of AI contributor, a platform builder and external agents, on what they wrote, how much they changed at once and how much of their work lasted, controlling for when the work was done. It relates the product’s rapid growth to the laws of software evolution. And it sets out ten guardrails, each dated and traced to evidence, arranged as a ladder of three observed levels and a proposed fourth that founders, platform vendors and educators can use.
The study is limited to how the software was built and kept under control. Evaluating the quality of the product’s outputs or its users’ outcomes would require a different design. Section 2 reviews related work, Section 3 describes the case and method, Section 4 reports the results, Section 5 presents the scaling ladder, Section 6 discusses the findings, Section 7 sets out threats to validity and Section 8 concludes.
2. Background and Related Work
2.1. End-User Programming, Low-Code Platforms and the Verification Problem
People who write software without being professional programmers have long been studied as end-user programmers: scientists writing analysis scripts, accountants building spreadsheet models, teachers making web pages [14]. Their central difficulty has always been verification. They can say what a program should do but have limited means of checking that it does it, so errors can persist unnoticed [14]. Research on end-user software engineering has argued for bringing testing, debugging and version control practices to these users in forms that fit their goals and skills [15]. Low-code development platforms widened the group further by replacing much of the code with visual models and configuration, so that people close to a business problem could build applications themselves [16,17].
Vibe coding goes further again. An AI app builder writes ordinary source code in mainstream languages and frameworks from conversational requests, and the person directing the work may never open a file. Karpathy’s original description concerned throwaway weekend projects [1], but the same tools are now used to build products that store user data and take payments. Brooks distinguished the accidental difficulties of software, such as syntax and tooling, from its essential difficulties: specifying, designing and testing the conceptual structure of a system [18]. AI builders remove much of the accidental work. Whether they remove any of the essential work, or simply move it to the person giving instructions, is an open question that this case can inform.
2.2. Vibe Coding in Practice
Research on vibe coding is young and mostly qualitative. Fawzy, Tahir and Blincoe reviewed 518 first-hand accounts from 101 practitioner sources and found that vibe coders value speed and accessibility, that most consider the resulting code fast but flawed, and that many skip testing or rely on AI tools to check the AI’s own output, which can leave them unable to debug what they built [3]. A multivocal review of 47 sources describes vibe coding as an iterative loop of generation, evaluation and revision in which developers’ work shifts toward specifying intent and validating output; it found productivity gains reported in 45% of sources and limited evidence on maintainability and on the effectiveness of safeguards [4].
Studies of practice add detail. Sarkar and Drosos analysed more than eight hours of think-aloud video and found that vibe coding shifts expertise toward managing context, evaluating generated code quickly and deciding when to intervene, with trust built through verification [5]. Chou and colleagues describe a spectrum from people who accept output without inspecting it to people who examine and adapt it, all coping with unpredictable generation that they experience as “rolling the dice” [6]. Pimenova and colleagues found that trust regulates where developers sit between delegation and co-creation, and that recurring pain points include specification, reliability, debugging and review burden [7]. A survey of 162 vibe coders, from non-coders to professionals, identified a “perception–action gap”: awareness of the risks of AI-generated code is widespread, but the ability to evaluate and verify it depends on experience [8]. At the professional end, experienced developers treat agents as collaborators, keep control over design and implementation and use their expertise to steer agent behaviour [9]. The present case speaks directly to the perception–action gap: it shows what a person without coding experience did to verify and control an AI-built system once paying users depended on it.
2.3. AI-Assisted Programming: Productivity, Quality and Security
Large language models trained on code made practical code generation possible [19], and their use across software engineering tasks has been mapped in systematic reviews [20]. Evidence on their effects is mixed. In a controlled experiment, developers given GitHub Copilot completed a JavaScript task 55.8% faster than a control group [10], and field data link acceptance of suggestions to perceived productivity [21,22]. In a randomised trial with experienced open-source developers working on their own repositories, allowing AI tools made them 19% slower, although they had expected a 24% speed-up and afterwards believed they had been 20% faster [11]. Observational studies show that programmers alternate between accelerating, when they know what they want, and exploring, when they do not [23]; that they find generated code hard to understand and debug [24]; that much of their time goes into verifying suggestions [25]; and that the main reasons for abandoning assistants are output that fails to meet requirements and difficulty controlling the tool [26].
Quality and security concerns recur. Assistants produce code that is often correct but sometimes subtly wrong [27]. About 40% of programs generated by an early assistant in security-relevant scenarios contained vulnerabilities [28], and participants with AI help wrote less secure code while believing it was more secure [29]. An industry analysis of 211 million changed lines reports that the share of moved, or refactored, code fell from 25% in 2021 to under 10% in 2024 while copied code rose from 8.3% to 12.3% [30], and the 2024 DORA report estimated that every 25% increase in AI adoption was associated with a 1.5% decrease in delivery throughput and a 7.2% reduction in delivery stability [31].
2.4. AI Coding Agents
Coding agents plan and carry out changes across several files, run commands and tests, and iterate on the results. They are evaluated on benchmarks built from real software issues [32], and interfaces designed for agents improve their performance [33]. Their contributions now appear in public repositories at scale: one dataset contains 932,791 pull requests opened by five agents across 116,211 repositories [34]. These studies examine contributions to projects maintained by professional developers who review the agents’ work. None that I found examines a commercial codebase in which every change came from AI and the only human in the loop could not read code.
2.5. Software Evolution, Maintenance and Startups
Lehman’s laws of software evolution hold that a system in real-world use must change continually, that its complexity increases unless work is done to reduce it, and that its functional content must keep growing to satisfy users [35,36]. A study of the Linux kernel found that it grew faster than the laws predict [37]. Parnas described how software “ages” as changes made by people who do not fully understand its design erode its structure [38], and analysis of change data from a large telecommunications system found evidence that code decays, with changes becoming harder over time [39]. The cost of such shortcuts has been described as technical debt, which must eventually be repaid [40,41]. Maintenance work itself has been classified since Swanson into corrective, adaptive and perfective activity [42], and commit messages have long been used to infer why changes were made [43,44]. Change metrics such as churn and the complexity of recent changes predict defects [45,46], and changes that later need fixing can be traced back through version history [47].
Software development in startups has its own literature. Startups work under severe time and resource pressure, favour speed over process and accumulate technical debt deliberately to find a market, then pay for it as they grow [48,49]. Vibe coding compresses this pattern: a single founder can now produce in weeks a codebase that would previously have needed a team. Whether Lehman’s dynamics apply, and at what speed, is a natural question for such a system.
2.6. The Gap This Study Addresses
The literature describes how practitioners feel about vibe coding, how professionals constrain agents, how AI assistance affects performance on bounded tasks and how software evolves under human maintenance. What is missing is a longitudinal account of a production system built by a non-programmer, showing who wrote what, how much direction it took, how stable the code was, how the codebase evolved, what went wrong at the platform level and what the person in charge did about it. Case study research suits such questions, in which the phenomenon cannot be separated from its context [50].
3. Case and Method
3.1. The Case
Curie (curie.id) is a web platform for researchers. It offers search across scholarly databases, automated literature reviews, a document editor that inserts citations to source papers, and an AI chat that answers questions from the research literature. It opened to the public on 7 July 2026 and had about 500 paying users by 6 October 2026, according to the company’s billing records.
Curie runs on a commercial hosted AI app builder, which is not named here for reasons of confidentiality. A user describes a change in the builder’s chat; the builder’s AI writes or edits the code, commits it to a git repository managed by the platform, and deploys a web frontend and serverless backend functions. The platform saves a checkpoint, a restorable snapshot of the whole application, for every builder turn, and any checkpoint can be deployed to production. From early August 2026 I also used external AI agents, ChatGPT and Claude, connected directly to the same application through the platform’s Model Context Protocol (MCP) interface. These agents can read files, edit code, run commands in the application’s development sandbox and create checkpoints.
I am a co-founder of Curie and the only person who directed changes to the production code during the study period; the builder’s conversation log shows no other sender. My training is in environmental engineering and I do not write code. I made no manual code edits. My role was to decide what to build, describe changes in plain language, test the results in the running product and decide what to keep, roll back or redo. This makes the case close to the pure form of vibe coding: every line of code was produced by an AI system, and the human could judge the work only through the behaviour of the software and the explanations the AI gave.
3.2. Design
The study is an exploratory, holistic single-case study whose unit of analysis is the production application, following the guidelines of Runeson and Höst for case study research in software engineering [50] and the community standards for repository mining and case studies [51]. I am both the subject and the researcher. This gives full access to data and context but creates a risk of self-serving interpretation. I address it by relying first on machine-generated records (commits, conversation metadata, checkpoint records and code), by fixing every measure and inference rule before examining the results, by testing differences statistically where the data allow, and by discussing threats to validity in Section 7 [52].
3.3. Data
Table 1 lists the data sources. All dates and times are in Western Indonesia Time (UTC+7), and day numbers count from the public launch on 7 July 2026 (day 1). The observation window runs from the platform’s initial commit at 23:20 on 6 July 2026 to the last commit analysed, at 06:21 on 17 October 2026, and spans 104 calendar days. The initial commit contained the platform’s starter template: 78 files, including 4,912 lines of code, most of them a third-party interface component library.
I extracted metadata only from the conversation log: who sent each message, when, its length in words, whether it carried an attachment, and the credits and tokens recorded for it. No instruction text is reported. The development notes were used only to date when a practice was adopted and why, never as evidence of how much work was done or how stable it was.
3.4. Attributing Changes to a Source
The platform labels commits by origin. Commits made by external agents carry a fixed system message that marks them as agent changes. I confirmed the label against my own session records for twelve agent commits made between 7 and 9 August. The platform records the same label for every connected agent and no other field in the commit metadata identifies the tool, so ChatGPT and Claude contributions are analysed together as one group. Commits produced by the builder AI carry either an AI-written title (for example, “Optimize version history storage and improve error handling”) or a generic default message. I checked five untitled commits spread across July against the checkpoint records, and each matched a builder checkpoint with an AI-generated name such as “Improved Inline Citation Features”. The switch from untitled to titled builder commits around 20 August coincides with no change in how I worked, so I treat it as a change in the platform’s labelling and attribute both kinds to the builder. Package updates, template files and platform-applied recommendations are grouped as housekeeping (64 commits). Two commits are classed as restore events and analysed separately: an unrequested builder commit on 7 August that reverted earlier changes, and the deliberate rollback of 16 October.
3.5. Measures
I counted only code files (JavaScript and TypeScript sources) and excluded built bundles and package lock files. Files were grouped into layers by path: frontend, serverless functions, shared server code, tests, a third-party interface component library and translations.
- Direction effort is the number of instructions I sent to the builder, their length in words, the share with attachments such as screenshots, and the credits and tokens the platform recorded.
- Active time is estimated from commit sessions: consecutive commits from the same source less than 60 minutes apart form a session, and session length runs from the first commit to the last. This is a lower bound, because work before the first commit of a session is not counted.
- Deployment cadence is the number of checkpoints deployed to production. The platform keeps only the latest deployment time for each checkpoint, so repeated deployments of the same version count once and the figure is a lower bound.
- Commit size is the number of code lines added plus removed. Differences between sources were tested with the Mann–Whitney U test, with the rank-biserial correlation as effect size.
- Re-touching is the share of file modifications made within 24 hours of the previous modification of the same file.
- Durability is the share of code lines added by a source that are still present in the final snapshot, attributing each surviving line with git blame, ignoring whitespace, to the commit that last wrote it; lines carried over from the initial commit are attributed to housekeeping. Because older code has had more time to be replaced, sources were compared within the same week and month of writing. The 302 commits discarded by the 16 October rollback are excluded and reported separately. Moved code counts as newly written, so durability is a conservative estimate for reorganised code.
- Change purpose classifies the 346 titled builder commits by keywords in their titles, in English and Indonesian, into new capability, refinement, corrective and restructuring changes, following earlier work on classifying changes from commit messages [43,44] and Swanson’s maintenance categories [42].
- Growth and complexity are measured on the weekly snapshots: total and test lines, number of files, mean file length, files over 1,000 lines, and decision points per thousand lines of non-test code. Decision points (conditional, loop, case and catch keywords and logical operators) are a simple proxy for McCabe’s cyclomatic complexity [53].
- Phases were identified from the weekly mix of commit sources before any other result was examined. Phase boundaries fall at the start of the week in which the mix changed.
3.6. Identifying Guardrails
I define a guardrail as a working practice adopted deliberately to reduce the risk of losing work, shipping a change that could not be verified, or misjudging what was running. A practice is included only if it can be traced to at least one piece of evidence outside my recollection: a code marker and its edit history, a commit, a platform object such as a named checkpoint or staging application, or a written rule in the contemporaneous notes. Each guardrail is dated by its first evidence.
3.7. Scope and Confidentiality
The paper reports platform-level and process-level events. Product-specific defects, anything concerning user data, and security configuration are outside its scope. The development platform is not named, and identifiers such as application IDs, function names and internal addresses are omitted. This keeps the focus on what other builders can learn about the method and avoids disclosing operational details of the product.
3.8. Use of AI in This Study
An AI model (Claude, Anthropic) wrote and ran the analysis scripts under my direction, inside the application’s development sandbox for the repository and through the platform’s web interface for the platform records, computed the statistics, produced the figures and prepared a first draft of the text, which I revised. I specified the research questions, measures, attribution rules and evidence rule for guardrails, supplied and checked the case facts, and checked the reported numbers against the analysis output. The analysis of the repository was run twice, on separate days, and produced identical results. The acknowledgments give further details.
4. Results
4.1. The Human Side of the Work (RQ1)
Every code change was generated by AI, but the work required a great deal of human direction (Table 2). Over the 104 days I sent the builder 1,940 instructions, a median of 19 words each (90th percentile 78 words), totalling 96,318 words; 439 of them (22.6%) carried attachments, which in my practice were usually screenshots of what I wanted changed. The builder produced 972 commits, about one for every two instructions. In my experience, instructions that produced no commit were usually questions, clarifications or retries of attempts I had rejected; the log metadata cannot confirm this. Of the builder’s 7,734 visible replies, 5,629 involved tool calls such as reading or writing files. The platform recorded 7,983 credits for 1,906 billed turns. The builder read 1.36 billion tokens of context and generated 10.8 million, a ratio of about 125 to 1, so most of the tokens processed were context, which includes the existing code and the conversation.
Commit sessions give a lower-bound estimate of active time. Builder work formed 196 sessions with a median length of 21 minutes, totalling 177 hours; agent work formed 73 sessions with a median of 36 minutes and 10 commits, totalling 75 hours. Together this is about 250 hours of active development, or roughly 17 hours a week alongside my other roles. Within a builder session the median gap between consecutive commits was 9.6 minutes. Work happened on 89 of the 104 days; 90.7% of builder and agent commits were made between 06:00 and 22:00, and 14.0% on weekends. I deployed 196 distinct versions to production on 50 different days.
4.2. Who Wrote the Code (RQ1)
The builder made 972 commits (41.6%) and the external agents 1,296 (55.5%); 64 were housekeeping and two were restore events. The two sources worked differently. Agent commits were much smaller: the median code-changing agent commit altered 21 lines, against 87 for the builder (Mann–Whitney U = 809,894.5, z = 15.87, p < 0.001; rank-biserial r = 0.40, a medium effect; Figure 2b). The median agent commit touched one file and the median builder commit three. Agents also made 96 commits that changed no code at all, mostly documentation and assets.
The sources concentrated on different layers (Figure 2a). The builder added most of the frontend lines (103.4 thousand, against 52.7 thousand by agents) and of the shared server code (20.0 thousand against 2.4 thousand). Agents added most of the lines in serverless functions (79.5 thousand against 14.2 thousand) and almost all of the test code (39.9 thousand against fewer than 100 lines). Lines added are a different measure from lines kept. In the final snapshot the builder accounted for 67.5% of frontend lines, 82.4% of serverless function lines and 93.9% of shared server code, while agents accounted for 99.4% of test lines (Table S1h). Most of the agent work on serverless functions did not survive: the 16 October rollback alone discarded 76.8 thousand added lines in that layer (Section 4.5). My notes record the reason: I turned to agents when a problem required reading across several files, running checks or tracing a fault through the server pipeline, and used the builder for interface changes that were easy to describe and to see. My impression, which the data cannot test because the platform does not distinguish the agents, was that the ChatGPT agents worked better than the Claude agents overall, particularly on interface design.
4.3. How the Work Evolved (RQ2)
The weekly mix of sources divides the period into four phases (Figure 1, Table 3). In the first four weeks after launch the builder made every change apart from housekeeping. This was the most intense period of direction: I sent between 159 and 307 instructions a week and deployed 21 to 34 versions a week. External agents joined in early August; in the week of 10 August they made 508 of 520 commits, the busiest week of the period, while my instructions to the builder fell to 26. From late August to late September the builder was again the only source of code, at a lower pace, and one week in early September saw no code changes. In the final phase agents returned on 2 October, this time also writing automated tests, and instructions to the builder fell to between 31 and 82 a week.
The purpose of the builder’s changes also shifted (Table 4). Of the 346 titled builder commits, 38.4% added new capability, 26.9% refined existing features, 18.5% were corrective and 13.0% restructured code; 3.2% could not be classified. Corrective changes made up 18.3% of titled commits in August and 13.5% in September, then rose to 33.3% in October, when the product had more features to maintain and the builder was used mainly for targeted fixes. Thirty-two of the 346 titles (9.2%) were in Indonesian and the rest in English.
4.4. Stability of the Code (RQ2)
Rapid re-touching was the norm for every source. About half of all edits to a code file came within 24 hours of the previous edit to the same file: 52.1% for agents, and 50.4% and 48.5% for titled and untitled builder commits. This pattern is consistent with the trial-and-correction rhythm that qualitative studies of vibe coding describe [5,6].
Of the 323,405 code lines added by the builder, the agents and housekeeping, 50.1% were still present at the end. Most of the loss came from the 16 October rollback, which discarded 110,879 added lines in one step (Section 4.5). Excluding that work, 76.3% of the remaining 212,526 added lines survived. On this basis, agent-written lines appear more durable than builder-written lines (83.4% against 72.4%). This comparison is misleading, because most surviving agent code was written in October and had less time to be replaced. Compared within the same month, builder code was more durable in August (73.9% against 62.5%) and agent code in October (92.6% against 90.4%). Compared within the six weeks in which both sources wrote substantial code, each source was more durable in three weeks (Figure 2c). I therefore find no consistent difference in durability between the two kinds of AI contributor. The durability of builder code rose from 67.3% for lines written in July to 82.3% for lines written in September, which also reflects age more than quality.
Figure 2.
(a) Code lines added by layer and source, excluding restore events. (b) Lines changed per code commit. (c) Share of added lines still present at the end, for the six weeks in which both sources wrote more than 500 lines, excluding the 302 commits discarded on 16 October.
Figure 2.
(a) Code lines added by layer and source, excluding restore events. (b) Lines changed per code commit. (c) Share of added lines still present at the end, for the six weeks in which both sources wrote more than 500 lines, excluding the 302 commits discarded on 16 October.

4.5. Growth and Complexity of the Codebase (RQ2)
The codebase grew quickly and almost linearly (Figure 3a). At the end of the first week it held 26,294 lines in 298 code files; at the end of the period it held 163,921 lines in 1,419 files, a 6.2-fold increase, or about 10,000 lines a week on average. Growth paused only in the quiet week of early September. The final snapshot held 107.1 thousand lines in the frontend, 30.2 thousand in server code (13.9 thousand in serverless functions and 16.3 thousand in shared modules), 19.5 thousand in tests (11.9%), 4.2 thousand in the third-party component library and 2.6 thousand in translations.
Complexity rose faster than size. Decision points per thousand lines of non-test code nearly doubled, from 115.9 to 224.6 (+94%), with the steepest rises in the first three weeks and again in the final phase (Figure 3b). Mean file length rose from 88 to 116 lines, and the number of files over 1,000 lines rose from one to seven (Figure 3c). Change concentrated in these large files: the most frequently edited file, the chat module, was changed in 172 commits and had grown to 2,681 lines. Restructuring commits made up 13.0% of titled builder changes, and complexity still rose.
No automated test was committed until 2 October, day 88. Over the next 13 days the test suite grew to 198 files, almost all written by agents. On 16 October at 06:20 I rolled back to an earlier version, publishing a named checkpoint taken before a large integration and returning the code to its state at 10:19 on 13 October. The rollback discarded 302 commits, 291 of them by agents, together with the 110,879 lines those commits had added (76.8 thousand in serverless functions and 20.2 thousand in tests), including 98 of the 198 test files and 11 of 101 serverless functions. Development resumed within half an hour, and 101 test files remained at the end of the period.
4.6. Platform-Level Problems (RQ3)
Three events in the period originated in how the platform itself behaved.
An unrequested commit that undid work. On 7 August at 10:37 a builder session produced a commit, with an AI-written title describing a quota reset, that touched five server and pipeline files, removing 121 lines and adding 25. It reverted changes made earlier that day. I had not asked for it, and its title gave no hint of what it did. The work was restored from the git history the same day.
Server functions running old code. In the same week, results from the review pipeline showed signs that old and new code were running together. Diagnosis showed that editing a shared server module did not reliably rebuild the serverless functions that imported it, and that already-running instances could keep serving the previous build. From the outside, a change could appear deployed when part of the system had not received it.
A rollback that removed tests. The 16 October rollback worked as designed: it returned the whole repository to a saved state. Because the test suite lived in the same repository, it was rolled back with the feature code. The safety net for code and the safety net for verification were the same object, so using one removed half of the other.
Each of these events was detected by watching the behaviour of the running product or the history of changes, which is the only view a non-programmer has.
4.7. Guardrails (RQ4)
Table 5 lists ten guardrails that meet the definition in Section 3.6, in order of first evidence. Six were first evidenced on a single day, 7 August; four of them (G3 to G6) directly answered the first two platform events. The records give a measure of use: external agents created 289 named checkpoints, the forced-rebuild marker was changed 16 times from 7 August onward, and the pipeline version constant 10 times.
Two further habits are worth recording, although they do not meet the evidence rule as cleanly. First, I asked agents to confirm a diagnosis in the code before changing anything. On at least one occasion an agent’s first explanation of a fault proved wrong when checked, and acting on it would have sent the work in the wrong direction. Second, when agents finished a change, I asked them to report what they had tested and what they had not, so that unverified work stayed visible.
5. A Scaling Ladder for Vibe-Coded Products
The guardrails accumulated as the product moved from a prototype to a service with paying users and from one AI contributor to several. Arranging them by the situation that called for them gives a ladder of four levels (Figure 4). Levels 1 to 3 describe what happened in this case; Level 4 is a proposal based on where the evidence points.
Level 1, prototype: one person and one builder AI. At this stage the main risk is losing work or being unable to undo a change that touched many files. Named checkpoints before risky work (G1) and a staging copy of the application (G2) are enough. Most vibe-coded projects can stay here as long as nobody depends on them.
Level 2, paying users. Once people rely on the product, the founder must be able to tell what is actually running and whether a change took effect. The practices at this level make the system’s state visible to someone who cannot read code: a version stamp in every output (G3), forced rebuilds after shared changes (G4), a history audit to catch changes nobody asked for (G5), one change per test run so that results can be attributed (G6), and failures that announce themselves (G7). In this case all five were adopted on the day of the first platform event, which suggests that a founder reaching paying users should put them in place before such an event happens.
Level 3, several AI contributors. With a builder and external agents both changing the code, behaviour can drift in ways no single session sees. Reports should describe only what the system recorded (G8), fixes should be tried on unrelated cases (G9), and the behaviours that matter should be written down as automated tests built from real examples (G10). The 16 October rollback adds one requirement that this case did not meet: tests must be stored where a rollback cannot reach them, for example in a separately synchronised repository, or protected by a check that refuses any restore that would delete them.
Level 4, a team. As a product grows, some responsibilities should move to people who can read code. I propose that engineers own architecture, security and data handling; that agent changes pass through code review [54], by a person or by a second agent with a different brief, and through continuous integration that runs the test suite on every change [55]; and that files above about 1,000 lines, where change and complexity concentrate, be split before they become hard for any contributor to change safely. The founder’s role at this level is to set and enforce the quality bar.
Box 1. Five lessons for founders building products with AI tools.
- Make every AI change reversible before you need to reverse it: save a named checkpoint before any change you cannot judge by looking at the product.
- Make the running system tell you its version. If you cannot read the code, a version stamp in every output is how you know what is live.
- Read your change history at the start of every work session. AI tools can commit changes nobody asked for.
- Change one thing per test. When results move, you need to know which change moved them.
- Write down what “working” means as automated tests built from real examples, and keep them where a rollback cannot delete them.
6. Discussion
6.1. The Founder’s Role: Directing and Checking
The most consistent pattern in this case is that the human contribution consisted of directing and verifying work. I wrote no code. I sent 1,940 instructions, deployed 196 versions, designed version stamps, rebuild markers, audit routines, testing rules and evaluation rules, and decided which of the AI’s explanations to trust. This mirrors the finding that experienced developers keep control of agents by steering and reviewing them [9], and it suggests one way across the perception–action gap reported for less experienced vibe coders [8]. In this case, a person who could not evaluate code directly kept control of an AI-built system through its observable behaviour, version stamps and logs, and working rules. This is the same remedy that end-user software engineering research proposed for spreadsheet authors and scientists [15], arriving by necessity in a new setting.
6.2. The Essential Work Remains
About 250 hours of active work and nearly 100,000 words of instructions went into this product. AI removed the accidental difficulty of writing code in Brooks’s sense, but the essential difficulty of deciding what the system should do, and checking that it does it, remained with me [18]. Instructions to the builder fell after July. Part of the direction moved to external agents, whose prompts were not counted, so the figures here understate the total direction the product received. Studies of AI assistants report that developers spend much of their time verifying suggestions [25] and that perceived speed can exceed real speed [11]. In this case the comparison is different: without AI, I would not have written the code at all. The realistic alternative was hiring a team, a comparison the data cannot make, but the effort recorded here sets a realistic expectation for founders who imagine that vibe coding is effortless.
6.3. Platform Behaviour Is a Major Source of Risk
Two of the three platform-level events, the unrequested reverting commit and the stale builds, came from the platform’s own mechanics, and the third came from using a platform feature as intended. For a founder who cannot read code, the platform is the whole development environment, so its behaviour matters as much as the code the AI writes. The evidence points to four improvements vendors could make. They could show which version of each server function is live, so that a founder need not invent a version stamp. They could label AI-initiated commits with what they changed, as well as an AI-written title, and flag commits that remove recent work. They could record which external agent made each change, which would let users and researchers compare agents. And they could warn before a restore deletes test files, or keep tests outside the scope of restores by default.
6.4. Two Kinds of AI Contributor
The builder and the agents played different roles. The builder made broad changes across several files, suited to interface work that I could see and judge immediately. The agents made small, targeted changes, added most of the lines written for the serverless functions and wrote nearly all the tests. Once timing was controlled, neither source’s code was consistently more durable. One possible explanation, which this case cannot test, is that the checks around the work mattered as much as which AI did it. The pattern supports a division of labour in which a builder handles what can be judged by eye and agents handle what must be checked by running it, with tests as the shared record of what “working” means. Tests arrived late, on day 88. Thin testing is common even among professional developers, who spend far less time testing than they believe [56], but for a founder who cannot read code, tests are also the only written specification of what the AI must preserve.
6.5. Rapid Growth and Rising Complexity
The codebase grew 6.2-fold in 14 weeks while decision points per thousand lines nearly doubled and the number of very large files rose from one to seven. This is Lehman’s continuing growth and increasing complexity [35,36] compressed into a single quarter, and it fits the startup pattern of deliberate technical debt taken on to reach the market [48,49]. Restructuring made up 13% of titled builder commits; whether AI-built systems receive less restructuring than human-built ones, as an industry analysis suggests [30], cannot be judged from one case. Code that no human fully understands is exposed to the aging that Parnas described [38] and to the decay that earlier studies measured in human-written systems [39], and in conventional codebases high churn and complex change histories predict defects [45,46]. The ladder’s Level 4 practices, especially splitting the largest files and running tests on every change, are one way to start paying down this debt [40,41]. Whether AI agents themselves can pay it down, by restructuring code they wrote, is an open question that this case cannot answer.
6.6. Implications for Founders, Educators, Platform Vendors and Researchers
For founders, the case shows that a non-programmer can take an AI-built product to paying users within weeks, and that in this case keeping it running depended on investing in checks. Level 2 practices should be in place by the day the first user pays, and a weekly glance at code size and the largest files is a cheap early warning of growing debt. For educators, it suggests that teaching people to direct AI builders should include version control concepts, testing from examples and reading system behaviour, even for students who will never write code by hand. This matters particularly in countries such as Indonesia, where domain experts in universities and small firms can now build the software their fields lack without waiting for scarce engineering capacity. For platform vendors, Section 6.3 lists concrete changes. For researchers, the version histories and conversation logs of AI-built products are an underused source; the measures used here are simple to compute and could be applied across many cases to test whether the patterns reported here are general.
7. Threats to Validity
Construct validity. Source attribution relies on platform commit labels. The external-agent label was checked against session records, and the attribution of untitled commits to the builder was checked against checkpoint records. Because the platform labels all external agents identically, ChatGPT and Claude contributions cannot be separated. Re-touching and durability are proxies. Re-touching cannot distinguish a correction from a planned follow-up, and git blame counts moved lines as new, which understates durability for reorganised code. Decision-point density is a rough proxy for cyclomatic complexity. Active time from commit sessions is a lower bound, and deployment counts are a lower bound because repeated deployments of the same checkpoint count once. The change-purpose classification relies on commit titles written by the builder, which may describe intentions more than effects. Prompts sent to external agents were not recorded, so direction effort is understated.
Internal validity. I am a co-founder of the product, the person who directed the work and the analyst. To limit bias, the quantitative results come from scripts run over complete records, the phase boundaries were set from the weekly source mix before other results were examined, durability was compared within the same period of writing, and every guardrail had to meet an evidence rule. The scope decision in Section 3.7 means that the paper reports platform-level and process-level events and not product-specific defects, so it should not be read as a complete account of everything that went wrong.
External validity. This is one product, one platform, one person directing the work and 104 days. I have an engineering degree, experience running technology-enabled companies and access to frontier AI models, none of which can be assumed for other non-programmers. Other app builders work differently. The ladder in Section 5 is offered as a hypothesis for other cases to test.
Reliability. The repository history begins on 6 July 2026, the day before public launch, so any earlier development is not observed; the initial commit held only the platform’s starter template. The repository analysis was repeated on a separate day with identical results. The measures are defined in Section 3.5, and the aggregated data behind the figures and tables are provided as Supplementary Material, so they can be recomputed and applied to other cases.
8. Conclusions
Curie was built entirely by AI tools under the direction of someone who does not write code, and it reached about 500 paying users within three months. Its records show the human side of that achievement: 1,940 instructions, roughly 250 hours of active work and 196 deployments. They also show a builder AI and external agents dividing the work, half of all edits returning to a file within a day, a codebase that grew six-fold while its complexity nearly doubled, tests arriving late and a rollback that showed why tests need protection of their own. Every problem described here was caught by watching what the system did, and most responses took the form of checks I designed.
In this case, vibe coding was enough to get a product into production. Keeping it there required practices that made the AI-built system observable and reversible to the person responsible for it, and it will require a plan for the complexity that rapid AI-driven growth leaves behind. The ten guardrails and the four-level ladder offer a starting point for founders about to make that move, and for the researchers, educators and platform vendors who support them.
Supplementary Materials
The following supporting information can be downloaded at the website of this paper posted on Preprints.org, Table S1: Aggregated data underlying the figures and tables, in eight sheets: weekly activity by source (S1a), weekly codebase snapshots (S1b), summary measures by source (S1c), durability by cohort (S1d), phases (S1e), change purpose by month (S1f), direction and effort measures (S1g), and code lines by layer and source, added and in the final snapshot (S1h) (Microsoft Excel file).
Author Contributions
Bintang Ekananda is the sole author. He built the product, conceived and designed the study, directed the analysis, wrote and revised the manuscript, and approved the final version.
Funding
This research received no external funding.
Institutional Review Board Statement
Not applicable. The study analysed the author’s own software repository and platform records and involved no other human participants and no user data.
Informed Consent Statement
Not applicable.
Data Availability Statement
The aggregated data underlying all figures and tables are provided in Table S1. Commit-level metadata and the analysis scripts are available from the author on reasonable request, subject to the confidentiality of the development platform. The product’s source code and the text of the conversation log are proprietary and are not shared.
Acknowledgments
I thank Faiq Rahman, co-founder of Curie, for his contributions to the product and for discussions that shaped this work. During the preparation of this manuscript, in October 2026, the author used Claude (Anthropic), a large language model, to write and run the analysis scripts, compute the statistics, produce the figures and prepare the first draft of the manuscript text and subsequent revisions under the author’s direction. The author designed the study, defined the measures and the evidence rule for guardrails, supplied and checked the case facts, and reviewed and edited all output, and takes full responsibility for the content of this publication.
Conflicts of Interest
The author is a co-founder and shareholder of Curie, the platform studied in this paper, and benefits financially from its success. He has no financial relationship with the vendor of the development platform, OpenAI or Anthropic beyond paying for their services as a customer.
References
- Karpathy, A. There’s a New Kind of Coding I Call “Vibe Coding”. Available online: https://x.com/karpathy/status/1886192184808149383 (accessed on 18 October 2026).
- Clark, L. “Vibe Coding” Named Word of the Year. Developers Everywhere Faceplant. Available online: https://www.theregister.com/2025/11/06/vibe_coding_escape_the_clutches/ (accessed on 18 October 2026).
- Fawzy, A.; Tahir, A.; Blincoe, K. Vibe Coding in Practice: Motivations, Challenges, and a Future Outlook – a Grey Literature Review. In Proceedings of the 48th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP 2026); 2026. [Google Scholar]
- Siddeeq, S.; Waseem, M.; Kemell, K.-K.; Saari, M.; Rasku, J.; Abrahamsson, P. Vibe Coding in Software Development: A Multivocal Literature Review. arXiv 2026, arXiv:2607.21652. [Google Scholar]
- Sarkar, A.; Drosos, I. Vibe Coding: Programming Through Conversation with Artificial Intelligence. arXiv 2025, arXiv:2506.23253. [Google Scholar]
- Chou, Y.-H.; Jiang, B.; Chen, Y.; Weng, M.; Jackson, V.; Zimmermann, T.; Jones, J. Building Software by Rolling the Dice: A Qualitative Study of Vibe Coding. Proc. ACM Softw. Eng. 2026, 3. [Google Scholar] [CrossRef]
- Pimenova, V.; Fakhoury, S.; Bird, C.; Storey, M.-A.; Endres, M. Good Vibrations? A Qualitative Study of Co-Creation, Communication, Flow, and Trust in Vibe Coding. arXiv 2025, arXiv:2509.12491. [Google Scholar]
- Fawzy, A.; Tahir, A.; Blincoe, K. From Prompting to Verification: How Experience Shapes Vibe Coding Practices. arXiv 2026, arXiv:2605.24521. [Google Scholar]
- Huang, R.; Reyna, A.; Lerner, S.; Xia, H.; Hempel, B. Professional Software Developers Don’t Vibe, They Control: AI Agent Use for Coding in 2025. arXiv 2025, arXiv:2512.14012. [Google Scholar]
- Peng, S.; Kalliamvakou, E.; Cihon, P.; Demirer, M. The Impact of AI on Developer Productivity: Evidence from GitHub Copilot. arXiv 2023, arXiv:2302.06590. [Google Scholar]
- Becker, J.; Rush, N.; Barnes, E.; Rein, D. Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. arXiv 2025, arXiv:2507.09089. [Google Scholar]
- Bird, C.; Rigby, P.C.; Barr, E.T.; Hamilton, D.J.; German, D.M.; Devanbu, P. The Promises and Perils of Mining Git. In Proceedings of the 6th IEEE International Working Conference on Mining Software Repositories (MSR 2009); 2009; pp. 1–10. [Google Scholar]
- Kalliamvakou, E.; Gousios, G.; Blincoe, K.; Singer, L.; German, D.M.; Damian, D. An in-Depth Study of the Promises and Perils of Mining GitHub. Empir. Softw. Eng. 2016, 21, 2035–2071. [Google Scholar] [CrossRef]
- Ko, A.J.; Abraham, R.; Beckwith, L.; Blackwell, A.; Burnett, M.; Erwig, M.; Scaffidi, C.; Lawrance, J.; Lieberman, H.; Myers, B.; et al. The State of the Art in End-User Software Engineering. ACM Comput. Surv. 2011, 43, 1–44. [Google Scholar] [CrossRef]
- Burnett, M.M.; Myers, B.A. Future of End-User Software Engineering: Beyond the Silos. In Proceedings of the Future of Software Engineering (FOSE 2014); 2014; pp. 201–211. [Google Scholar]
- Bock, A.; Frank, U. Low-Code Platform. Bus. Inf. Syst. Eng. 2021, 63, 733–740. [Google Scholar] [CrossRef]
- Di Ruscio, D.; Kolovos, D.; de Lara, J.; Pierantonio, A.; Tisi, M.; Wimmer, M. Low-Code Development and Model-Driven Engineering: Two Sides of the Same Coin? Softw. Syst. Model. 2022, 21, 437–446. [Google Scholar] [CrossRef]
- Brooks, F.P. No Silver Bullet: Essence and Accidents of Software Engineering. Computer 1987, 20, 10–19. [Google Scholar] [CrossRef]
- Chen, M.; et al. Evaluating Large Language Models Trained on Code. arXiv 2021, arXiv:2107.03374. [Google Scholar]
- Hou, X.; Zhao, Y.; Liu, Y.; Yang, Z.; Wang, K.; Li, L.; Luo, X.; Lo, D.; Grundy, J.; Wang, H. Large Language Models for Software Engineering: A Systematic Literature Review. ACM Trans. Softw. Eng. Methodol. 2024, 33, 1–79. [Google Scholar] [CrossRef]
- Ziegler, A.; Kalliamvakou, E.; Li, X.A.; Rice, A.; Rifkin, D.; Simister, S.; Sittampalam, G.; Aftandilian, E. Measuring GitHub Copilot’s Impact on Productivity. Commun. ACM 2024, 67, 54–63. [Google Scholar] [CrossRef]
- Bird, C.; Ford, D.; Zimmermann, T.; Forsgren, N.; Kalliamvakou, E.; Lowdermilk, T.; Gazit, I. Taking Flight with Copilot. ACM Queue 2022, 20, 35–57. [Google Scholar] [CrossRef]
- Barke, S.; James, M.B.; Polikarpova, N. Grounded Copilot: How Programmers Interact with Code-Generating Models. Proc. ACM Program. Lang. 2023, 7, 85–111. [Google Scholar] [CrossRef]
- Vaithilingam, P.; Zhang, T.; Glassman, E.L. Expectation Vs. Experience: Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In Proceedings of the CHI Conference on Human Factors in Computing Systems Extended Abstracts; 2022. [Google Scholar]
- Mozannar, H.; Bansal, G.; Fourney, A.; Horvitz, E. Reading Between the Lines: Modeling User Behavior and Costs in AI-Assisted Programming. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI 2024); 2024; pp. 1–16. [Google Scholar]
- Liang, J.T.; Yang, C.; Myers, B.A. A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE 2024); 2024; pp. 1–13. [Google Scholar]
- Moradi Dakhel, A.; Majdinasab, V.; Nikanjam, A.; Khomh, F.; Desmarais, M.C.; Jiang, Z.M. GitHub Copilot AI Pair Programmer: Asset or Liability? J. Syst. Softw. 2023, 203, 111734. [Google Scholar] [CrossRef]
- Pearce, H.; Ahmad, B.; Tan, B.; Dolan-Gavitt, B.; Karri, R. Asleep at the Keyboard? Assessing the Security of GitHub Copilot’s Code Contributions. In Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP); 2022; pp. 754–768. [Google Scholar]
- Perry, N.; Srivastava, M.; Kumar, D.; Boneh, D. Do Users Write More Insecure Code with AI Assistants? In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS 2023); 2023; pp. 2785–2799. [Google Scholar]
- GitClear AI Copilot Code Quality: 2025 Look Back at 12 Months of Data. Available online: https://www.gitclear.com/ai_assistant_code_quality_2025_research (accessed on 18 October 2026).
- Google Cloud DORA Accelerate State of DevOps Report 2024. Available online: https://cloud.google.com/blog/products/devops-sre/announcing-the-2024-dora-report (accessed on 18 October 2026).
- Jimenez, C.E.; Yang, J.; Wettig, A.; Yao, S.; Pei, K.; Press, O.; Narasimhan, K. SWE-Bench: Can Language Models Resolve Real-World GitHub Issues? In Proceedings of the Twelfth International Conference on Learning Representations (ICLR 2024); 2024. [Google Scholar]
- Yang, J.; Jimenez, C.E.; Wettig, A.; Lieret, K.; Yao, S.; Narasimhan, K.; Press, O. SWE-Agent: Agent-Computer Interfaces Enable Automated Software Engineering. arXiv 2024, arXiv:2405.15793. [Google Scholar]
- Li, H.; Zhang, H.; Hassan, A.E. AIDev: Studying AI Coding Agents on GitHub. arXiv 2026, arXiv:2602.09185. [Google Scholar]
- Lehman, M.M. Programs, Life Cycles, and Laws of Software Evolution. Proc. IEEE 1980, 68, 1060–1076. [Google Scholar] [CrossRef]
- Lehman, M.M.; Ramil, J.F.; Wernick, P.D.; Perry, D.E.; Turski, W.M. Metrics and Laws of Software Evolution: The Nineties View. In Proceedings of the Fourth International Software Metrics Symposium; 1997; pp. 20–32. [Google Scholar]
- Godfrey, M.W.; Tu, Q. Evolution in Open Source Software: A Case Study. In Proceedings of the International Conference on Software Maintenance (ICSM 2000); 2000; pp. 131–142. [Google Scholar]
- Parnas, D.L. Software Aging. In Proceedings of the 16th International Conference on Software Engineering (ICSE 1994); 1994; pp. 279–287. [Google Scholar]
- Eick, S.G.; Graves, T.L.; Karr, A.F.; Marron, J.S.; Mockus, A. Does Code Decay? Assessing the Evidence from Change Management Data. IEEE Trans. Softw. Eng. 2001, 27, 1–12. [Google Scholar] [CrossRef]
- Cunningham, W. The WyCash Portfolio Management System. In Proceedings of the OOPSLA ’92 Addendum to the Proceedings on Object-Oriented Programming Systems, Languages, and Applications; 1992; pp. 29–30. [Google Scholar]
- Kruchten, P.; Nord, R.L.; Ozkaya, I. Technical Debt: From Metaphor to Theory and Practice. IEEE Softw. 2012, 29, 18–21. [Google Scholar] [CrossRef]
- Swanson, E.B. The Dimensions of Maintenance. In Proceedings of the 2nd International Conference on Software Engineering (ICSE 1976); 1976; pp. 492–497. [Google Scholar]
- Mockus, A.; Votta, L.G. Identifying Reasons for Software Changes Using Historic Databases. In Proceedings of the International Conference on Software Maintenance (ICSM 2000); 2000; pp. 120–130. [Google Scholar]
- Hindle, A.; German, D.M.; Holt, R.C. What Do Large Commits Tell Us? A Taxonomical Study of Large Commits. In Proceedings of the 2008 International Working Conference on Mining Software Repositories (MSR 2008); 2008; pp. 99–108. [Google Scholar]
- Nagappan, N.; Ball, T. Use of Relative Code Churn Measures to Predict System Defect Density. In Proceedings of the 27th International Conference on Software Engineering (ICSE 2005); 2005; pp. 284–292. [Google Scholar]
- Hassan, A.E. Predicting Faults Using the Complexity of Code Changes. In Proceedings of the 31st International Conference on Software Engineering (ICSE 2009); 2009; pp. 78–88. [Google Scholar]
- Śliwerski, J.; Zimmermann, T.; Zeller, A. When Do Changes Induce Fixes? In Proceedings of the 2005 International Workshop on Mining Software Repositories (MSR 2005); 2005; pp. 1–5. [Google Scholar]
- Paternoster, N.; Giardino, C.; Unterkalmsteiner, M.; Gorschek, T.; Abrahamsson, P. Software Development in Startup Companies: A Systematic Mapping Study. Inf. Softw. Technol. 2014, 56, 1200–1218. [Google Scholar] [CrossRef]
- Giardino, C.; Paternoster, N.; Unterkalmsteiner, M.; Gorschek, T.; Abrahamsson, P. Software Development in Startup Companies: The Greenfield Startup Model. IEEE Trans. Softw. Eng. 2016, 42, 585–604. [Google Scholar] [CrossRef]
- Runeson, P.; Höst, M. Guidelines for Conducting and Reporting Case Study Research in Software Engineering. Empir. Softw. Eng. 2009, 14, 131–164. [Google Scholar] [CrossRef]
- Ralph, P.; et al. Empirical Standards for Software Engineering Research. arXiv 2020, arXiv:2010.03525. [Google Scholar]
- Wohlin, C.; Runeson, P.; Höst, M.; Ohlsson, M.C.; Regnell, B.; Wesslén, A. Experimentation in Software Engineering; Springer: Berlin, 2012; ISBN 978-3-642-29044-2. [Google Scholar]
- McCabe, T.J. A Complexity Measure. IEEE Trans. Softw. Eng. 1976, SE-2, 308–320. [Google Scholar] [CrossRef]
- Bacchelli, A.; Bird, C. Expectations, Outcomes, and Challenges of Modern Code Review. In Proceedings of the 35th International Conference on Software Engineering (ICSE 2013); 2013; pp. 712–721. [Google Scholar]
- Hilton, M.; Tunnell, T.; Huang, K.; Marinov, D.; Dig, D. Usage, Costs, and Benefits of Continuous Integration in Open-Source Projects. In Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering (ASE 2016); 2016; pp. 426–437. [Google Scholar]
- Beller, M.; Gousios, G.; Panichella, A.; Zaidman, A. When, How, and Why Developers (Do Not) Test in Their IDEs. In Proceedings of the 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015); 2015; pp. 179–190. [Google Scholar]
Figure 1.
(a) Commits per week by source, with the four phases and three events discussed in the text. (b) Instructions sent by the author to the builder AI per week. (c) Distinct versions deployed to production per week.
Figure 1.
(a) Commits per week by source, with the four phases and three events discussed in the text. (b) Instructions sent by the author to the builder AI per week. (c) Distinct versions deployed to production per week.

Figure 3.
Growth of the codebase, measured at the last commit of each week. (a) Total and test lines. (b) Decision points per thousand lines of non-test code. (c) Number of files longer than 1,000 lines.
Figure 3.
Growth of the codebase, measured at the last commit of each week. (a) Total and test lines. (b) Decision points per thousand lines of non-test code. (c) Number of files longer than 1,000 lines.

Figure 4.
A four-level ladder for vibe-coded products. Each level adds practices to those below it.

Table 1.
Data sources.
| Source | Content | Used for |
| Git history of the production application | 2,334 commits with timestamps, messages and per-file lines added and removed | Work split, phases, commit size, re-touching, sessions, change purpose |
| Code snapshots | Final snapshot (1,419 code files, 163,921 lines) with line-level origin from git blame; one snapshot at the end of each week | Durability, layers, growth and complexity |
| Builder conversation log | 9,675 messages, of which 1,940 are the author’s instructions; message metadata, credits and token counts | Direction effort and cost |
| Platform checkpoint records | 2,457 checkpoints, 196 of them deployed to production; 289 created by external agents | Deployment cadence, attribution, guardrails |
| Code markers | A forced-rebuild marker and a pipeline version constant, with their edit histories | Dating guardrails |
| Contemporaneous development notes | Session summaries recording decisions, rules and their reasons | Context and dating of guardrails only |
| Billing records | Number of paying users on 6 October 2026 | Case description |
Table 2.
Code changes by source, 6 July to 17 October 2026.
| Measure | Builder AI | External agents | Housekeeping |
| Commits | 972 (41.6%) | 1,296 (55.5%) | 64 (2.7%) |
| Commits that changed no code file | 5 | 96 | 52 |
| Code lines added | 138,975 | 177,851 | 6,579 |
| Lines changed per code commit: median (interquartile range) | 87 (29–231) | 21 (6–79) | – |
| Median files per commit | 3 | 1 | 1 |
| File edits within 24 h of the previous edit to the same file | 50.4% (titled), 48.5% (untitled) | 52.1% | 29.4% |
| Added lines still present at the end, all work | 72.3% | 31.5% | 85.5% |
| Added lines still present at the end, excluding rolled-back work | 72.4% | 83.4% | 85.5% |
| Share of the 163,921 lines in the final codebase | 61.3% | 34.1% | 3.4% |
Note: the two restore events are not shown; they account for the remaining 1.1% of lines in the final codebase. Housekeeping lines in the final codebase come mostly from the platform’s starter template. Durability differences between sources are not consistent once the timing of work is controlled (Section 4.4).
Table 3.
Phases of development.
| Phase | Dates (2026) | Commits | Builder | Agents | Instructions | Deploys | Lines added | Main activity |
| 1. Builder only | 6 Jul–2 Aug | 539 | 518 | 0 | 1,013 | 105 | 74,663 | Core product |
| 2. Agents join | 3–23 Aug | 876 | 159 | 711 | 342 | 32 | 45,147 | Server pipeline; first guardrails |
| 3. Builder again | 24 Aug–27 Sep | 233 | 205 | 0 | 427 | 26 | 40,166 | Interface and features |
| 4. Agents and tests | 28 Sep–17 Oct | 686 | 90 | 585 | 158 | 33 | 165,372 | Features, tests; rollback |
Note: commits include housekeeping and restore events. Phase 4 lines added include work later discarded by the 16 October rollback.
Table 4.
Purpose of titled builder commits by month.
| Month (2026) | Titled commits | New capability | Refinement | Corrective | Restructuring | Unclassified |
| July | 15 | 2 | 10 | 3 | 0 | 0 |
| August | 142 | 62 | 40 | 26 | 14 | 0 |
| September | 141 | 59 | 29 | 19 | 26 | 8 |
| October | 48 | 10 | 14 | 16 | 5 | 3 |
| Total | 346 | 133 (38.4%) | 93 (26.9%) | 64 (18.5%) | 45 (13.0%) | 11 (3.2%) |
Note: most builder commits in July were untitled and cannot be classified by purpose.
Table 5.
Guardrails adopted during the study period.
| No. | Guardrail | What prompted it | Evidence | First evidence | Level |
| G1 | Deliberately named checkpoint before and after each unit of risky work | Changes that touch many files at once | 289 checkpoints created by agents, alongside the builder’s automatic ones | 6 August | 1 |
| G2 | Staging copy of the application for risky changes | Need to try changes without touching paying users | Staging application in platform records | Early August (approx.) | 1 |
| G3 | Pipeline version and load time printed in every output | Uncertainty about which code produced a result | Version constant, 10 edits | 7 August | 2 |
| G4 | Forced-rebuild marker changed with every shared-module edit | Server functions serving stale builds | Marker in the main worker, 16 edits | 7 August | 2 |
| G5 | History audit at the start of every work session | Unrequested commit that reverted work | Written rule in notes | 7 August | 2 |
| G6 | One change set per validation run | Results that could not be attributed to a single change | Written rule in notes | 7 August | 2 |
| G7 | External services must fail visibly, never silently | Reliance on several external data services with usage limits | Typed error handling in commits | 7 August | 2 |
| G8 | Generated reports state only what the system recorded | Need for reports that a user can audit | Report text keyed to run records, in commits | 7 August | 3 |
| G9 | Fixes must work on unrelated cases, never only on the test case | Risk of tuning a change to the example used to test it | Written rule in notes | September (approx.) | 3 |
| G10 | Automated tests written from real cases | Repeated manual checking of the same behaviours | 199 test files created, 101 remaining | 2 October | 3 |
Note: G2 and G9 rest on undated notes, so their dates are approximate.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.