Section 1
Confirmed Descriptive Results
| Measure | Result | Interpretation |
|---|---|---|
| Authored messages | 1,711 | 1,585 in the general group and 126 in the builders group; includes short replies and media placeholders. |
| Visible senders | 57 | Distinct authored-message labels across the combined corpus; not a membership count. |
| Media-only exclusions | 146 | Entries marked "Media omitted" were not interpreted or coded. |
| Conservative build inventory | 14 | Distinct applications, prototypes or substantial automations described in enough text to classify. |
| Firsthand operational origin | 10 / 14 | Builder explicitly linked the artefact to their own work, business process or recurring frustration. |
| Reported beyond concept stage | 8 / 14 | Reported as deployed, in internal use, in beta or used by customers; a URL, deployment or use context was explicitly stated. |
| Paying-customer claim | 1 | The product founder reported ten paying customers; a participant claim, not independently verified. |
| Validation episodes | 6 | Tester recruitment, beta feedback, ideal-customer definition, case studies and problem intensity. |
| Integration barrier episodes | 7 | WhatsApp policy, Simpro API access, Xero/Fergus/Gmail connections and refresh-token issues. |
| Peer-learning episodes | 12 | Tool advice, demos, critiques, implementation help, Zooms, coffee meetings and hackathon planning. |
| Security / reliability episodes | 4 | Data access risk, production security, platform restrictions and quote-error risk. |
Table S1: Confirmed descriptive results across the combined corpus
Counting Rule
Repeated messages about one product or topic are consolidated into one episode. Episode boundaries were drawn by product or topic continuity rather than by date: an exchange returning to the same build across multiple days (e.g., G05, spanning 5–16 July) is one episode, while distinct builds discussed in the same thread are separate episodes. Join/leave notices, spam, duplicated links, jokes without analytical content, deleted messages and media-only posts are excluded. "Confirmed" means confirmed as present in the transcript — not independently verified in the external world.
Section 2
Codebook
Ten codes were applied at episode level. Inclusion and exclusion rules are stated explicitly so that a second reviewer can apply the same framework and reach materially similar classifications.
| Code | Definition | Include when… | Exclude when… |
|---|---|---|---|
| SELF | Build-for-self | The builder says the product solves their own workflow, cost or frustration. | A generic market opportunity is asserted without personal use. |
| DOMAIN | Domain knowledge | Trade/construction experience shapes requirements, workflow or product judgment. | Industry is named only as a target market. |
| RAPID | Reduced build barrier | AI is linked to faster, cheaper or non-traditional software production. | Speed/cost is assumed but not mentioned or demonstrated. |
| PEER | Peer learning | Members explain, critique, demo, compare tools or arrange feedback. | Pure promotion without exchange. |
| VALID | Testing / validation | Difficulty finding users, testers, evidence of demand, or a hair-on-fire problem. | General product enthusiasm. |
| SALES | Commercialisation | Pricing, customers, promotion, partnerships, leads or conflicts of interest. | Building only for personal use with no selling discussion. |
| INTEG | Integration / data access | WhatsApp, email, Xero, Gmail, MCPs, files or multi-platform data flows. | A standalone interface with no data dependency. |
| SEC | Security / reliability | Architecture, security, dependability or production risk is explicitly raised. | Researcher infers risk without transcript support. |
| TOOL | Pragmatic model choice | A builder selects or sequences models/tools for different purposes. | A model is merely named. |
| MOD | Community infrastructure | Moderation, spam removal, group rules, demos or access design. | Ordinary conversation. |
Table S2: Codebook — definitions with inclusion and exclusion rules
Code Notes
Two boundary observations from independent review. First, VALID and SALES frequently co-occur: tester recruitment is often simultaneously a validation problem and a commercialisation problem, and episodes carrying both codes should not be read as double-counting. Second, INTEG is deliberately broad — it bundles technical access (APIs, refresh tokens), legal and policy access (WhatsApp automation restrictions), and commercial access (vendor pricing tiers). These are distinct failure modes sharing one code because participants experienced them as a single barrier: the system could not reach the data it needed.
Section 3
Episode-Level Evidence Matrix
Nineteen consolidated episodes were coded across the two groups. "G" episodes originate in the general community; "B" episodes in the builders group. Each is traceable by date, speaker pseudonym and topic in the source export.
| ID / period | Episode | Codes | Evidence status |
|---|---|---|---|
| G01 23–27 Jun | Members compare Lovable, Claude Code and Codex; low-skill web deployment and persistent project context are explained. | RAPID, TOOL, PEER | Direct peer instruction and self-reported builds. |
| G02 23 Jun | Australian contractor asks how to track job costs for 13 employees before an interstate contract; a member offers a call. | SELF, DOMAIN, PEER | Direct problem articulation and informal support. |
| G03 25–26 Jun | Agency founder describes marketing and sales automations; peers critique positioning and target-market clarity. | SELF, RAPID, SALES, PEER | Direct customer-discovery and positioning exchange. |
| G04 27 Jun–16 Jul | AI learning app and recurring show-and-tell structure emerge from observed community needs. | SELF, RAPID, PEER, MOD | Prototype and community infrastructure reported. |
| G05 5–16 Jul | Construction record platform seeks beta testers, case studies and feedback while confronting WhatsApp constraints. | SELF, DOMAIN, VALID, INTEG, SALES | Repeated direct recruitment and platform discussion. |
| G06 6 Jul | Electrician reports AI-built website and solar estimator connected to Google Solar, Zapier and Fergus. | SELF, DOMAIN, RAPID, INTEG | Internal/business use and jobs attributed by participant. |
| G07 12 Jul | Practitioners report building a business PWA, AI bookkeeper and real-time accounting dashboard during ordinary work. | SELF, DOMAIN, RAPID, INTEG | Direct self-report; deployment outcomes vary. |
| G08 22 Jul | Plumber describes email triage, gas-certificate app, website and automated ads; another practitioner reports a mobile price book blocked by Simpro pricing/API limits. | SELF, DOMAIN, RAPID, INTEG | Direct artefact descriptions and barrier evidence. |
| G09 27–29 Jul | Simpro API restrictions block an intended Claude connection; peers propose alternatives and report similar API problems. | INTEG, PEER, VALID | Direct correspondence reproduced in chat; external policy not independently checked. |
| G10 29 Jul | Relay scheduled-task dashboard is built overnight and released for community feedback. | SELF, RAPID, VALID, PEER | Public link and feedback request supplied. |
| G11 29 Jul | Plumber tests plan-PDF extraction for quoting; peers suggest turning it into a scheduled agent. | SELF, DOMAIN, RAPID, PEER | Direct internal-use workflow; still testing. |
| G12 29 Jul | Concrete QA discussion connects a manual record burden to a new anomaly-flagging app and produces a volunteer tester. | DOMAIN, VALID, PEER, RAPID | Direct problem discovery and tester match. |
| G13 5 Aug | Builder reports an estimating app used on own jobs; group debates custom price books, accuracy and risk of AI-generated quotes. | SELF, DOMAIN, SEC, VALID | Direct use claim and explicit negative/risk case. |
| B01 1–2 Jul | Builders group details the record platform and an integrated project-management scaffold. | SELF, DOMAIN, RAPID, INTEG, SALES | Deeper product descriptions; stages remain self-reported. |
| B02 2 Aug | Builder contrasts an earlier £15k quote with AI-assisted builds completed initially in hours and refined over weeks. | RAPID | Participant recollection; cost and elapsed time unverified. |
| B03 2 Aug | Founder reports a globally launched AI administration product and ten paying customers. | RAPID, SALES | Participant claim; customer count unverified. |
| B04 2 Aug | Members discuss weak sales confidence, employer conflict, scarce testers and the need for a "hair on fire" problem. | VALID, SALES, PEER | Direct evidence of perceived downstream barriers. |
| B05 2–4 Aug | Spam and cold promotion trigger removal/reporting and discussion of bot/platform rules. | MOD | Direct evidence that moderation is operational infrastructure. |
| B06 11 Aug | Founder seeks trades businesses using Xero and Gmail to test AI-native bookkeeping. | VALID, INTEG, SALES | Direct recruitment request; outcomes unknown. |
Table S3: Episode-level evidence matrix
Section 4
Build-Level Comparison
Nine builds were described in sufficient detail to compare origin, reported stage, core dependency and the specific reality barrier each faced. Five additional builds met the minimum inventory threshold but lacked sufficient detail for meaningful cross-build comparison; they remain in the count of fourteen and in the episode matrix above.
| Build | Origin | Stage | Core dependency | Reality barrier |
|---|---|---|---|---|
| WhatsApp site diary | Firsthand construction workflow | Working pages; automation incomplete | WhatsApp/API, email, files | Legitimate data access, partners, testing and sales |
| Integrated project app | Explicitly built for self | HTML/PWA scaffold | Scheduling, files, Gantt, external software/MCP | Refinement, user adoption and productisation |
| AI-native admin / bookkeeper | Trades target; founder-built | Reported live globally | Xero and Gmail | Qualified testers, traction, architecture/security |
| Marketing / sales work layer | Grew from founder's agency | Early customers / stealth reported | Websites, reviews, CRM workflows | Positioning and repeatable market segment |
| Solar estimator website | Electrician's own business | Deployed and used for enquiries | Google Solar, Zapier, Fergus | Integration upkeep and attribution |
| Gas certificate app | Plumber's own compliance work | Internal use reported | Google scripts / business data | Generalisation and product validation |
| Relay task dashboard | Founder's own task visibility need | Public early version | Claude and ChatGPT scheduled tasks | Feedback, usefulness and maintenance |
| Concrete QA app | Manual QA and mix-design records | Prototype; tester identified | PDF/photo ingestion and anomaly rules | Domain validation and reliable detection |
| Estimating app | Builder's own quoting workflow | Used internally; public demo linked | Company price books and back-costing data | Accuracy, liability and firm-specific configuration |
Table S4: Build-level comparison
Section 5
What the Evidence Supports
Supported
- Ten of fourteen conservatively inventoried builds were explicitly tied to firsthand work or business problems.
- Generative AI enabled practitioners to create interfaces and early products without a conventional development process.
- Peer exchange extended beyond encouragement into implementation help, product critique, tester matching, demonstrations and tool comparison.
- The bottleneck shifted from initial construction toward validation, integration, trust, sales and production readiness.
- Model choice was pragmatic rather than ideological: builders selected different tools for speed and refinement.
- Moderation and recurring events were necessary to protect and extend the usefulness of the open community.
Not Supported
- A claim that the group represents all tradespeople, construction businesses, or countries.
- Independent verification of reported customers, savings, build times, product quality or external usage.
- A causal claim that AI alone produced commercial success.
- A conclusion that professional software engineering, architecture or security is no longer required.
- Any conclusion based on screenshots, demonstrations, voice notes or other entries marked "Media omitted".
Section 6
Recommended Wording Discipline
The coding pass produced a set of substitutions between the wording an enthusiastic first draft reaches for and the wording the evidence actually carries. These substitutions govern the language used in the main paper.
| Tempting wording | Evidence-based replacement |
|---|---|
| "Builders frequently started with their own problems." | "Ten of fourteen conservatively inventoried builds were explicitly connected to a problem in the builder's own work or business." |
| "AI dramatically reduces development cost." | "One participant contrasted a previous £15,000 quote with AI-assisted prototypes built initially in hours; this is a self-reported comparison." |
| "The main constraint is no longer building." | "Across the coded episodes, testing, integration, reliability, trust and sales repeatedly appeared after an initial artefact could be produced." |
| "Tradies are becoming software developers." | "Several domain practitioners used AI coding tools to create working software artefacts despite no conventional development process being described." |
| "Multiple products reached the market." | "Eight of fourteen inventoried builds were reported as deployed, in internal use, in beta or used by customers; these stages were not independently verified." |
Table S5: Claim discipline applied to the main paper
Section 7
Next Validation Work & Audit Trail
The combined first pass is complete. Further work would increase research confidence rather than fill a missing corpus:
- A human reviewer could independently code a stratified sample using the codebook above; the AI-performed re-coding pass recorded below currently serves this function.
- Invite recognisable product owners to confirm descriptions and self-reported stages.
- Retain explicit negative cases, including API failure, beta-recruitment difficulty and quote-risk concerns.
- Interview selected builders to distinguish prototypes, internal tools, active users and paying customers.
- Repeat the analysis after a defined follow-up period to observe continuation or abandonment.
Second-Coder Check
An independent re-coding pass was performed in August 2026 by an AI model (Grok, xAI) given the complete raw exports of both groups and no access to the coding rationale. It independently re-coded all nineteen episodes and the build inventory. Results: full agreement on episode presence and primary codes of approximately 85–90 percent; partial disagreement on secondary code weighting or episode boundaries of approximately 10–12 percent; and two to three items flagged as potential over- or under-claims, including one borderline entry in the fourteen-build inventory, which is retained with an explicit maturity-range caveat in the main paper. Key descriptive statistics were independently verified, including an exact match on the 146 media-only exclusions. This check is disclosed as AI-performed, consistent with the diligence statement below. The codebook and boundary notes above are written so that a human reviewer could independently apply them and reach materially similar classifications; no such pass is currently scheduled.
Audit Trail
Each episode is traceable by date, speaker pseudonym and topic in the source export. Direct identifiers and real group names are omitted. The unchanged export remains the source record; another reviewer should be able to apply this codebook and reach materially similar classifications.
Before Release
An AI-performed second-coder check and challenge review is complete and recorded below · follow-up evidence distinguishing prototype, active internal use, paying customers and discontinued projects · Of the three participant decisions: (1) research consent has been obtained from the groups' founder and organiser, who has also consented to being identified by name and business; (2) no other participant is identified — group names remain pseudonymous and build descriptions are non-attributable, so further identification permissions are not required under the current design; (3) accuracy confirmation remains open for any build description that becomes attributable through the founder's identification, and should be confirmed with him directly before release. Any future move to name the groups themselves would reopen decisions (2) and (3) for every recognisable builder.
Return to the Paper
The Reality Barrier
What happens when tradies start building with AI — the argument these codes support, and the argument they refuse to support.
← Read Paper 2.2