The Trigger
TokiQR's voice QR codes were working. Record, encode, convert to QR, play back. The entire flow was stable. What was needed next was a pipeline to deposit these recordings with the National Diet Library (NDL).
We had been making deposits before. But as operations continued, constraints that the original design could not handle began surfacing one after another. A separate repository for each customer, PDFs stored as binaries in git, client-side builds. Every decision that seemed rational during development became a liability in the operations phase.
Over two days, I pushed roughly sixty pull requests across three repositories. This is a record of what happened.
Constraints, in the Order They Appeared
GitHub Bandwidth Limits
The first wall was GitHub's API bandwidth limit. The design called for creating a newsletter repository per customer and storing PDFs there, but files over 1MB hit rate limits. One or two deposits were fine, but submitting multiple newsletters in succession quickly exhausted the quota.
QR Codes Embedded in PDFs Were Unreadable
Newsletter PDFs contain QR codes that store the audio data — essential for restoration after deposit. But when we tried to read QR codes back from the PDFs, they could not be reliably detected.
Digging into the cause revealed a chain of constraints. PDFs generated by jsPDF were bloated, and since git cannot diff binaries, the repository swelled with every update. Worse, the QR scanning resolution was insufficient. Rendering a Version 40 QR code at 800px produced only about 2.7 pixels per module. The jsQR library needs a minimum of 3–4 pixels per module for reliable detection — the detection rate was not viable for production use.
We tested method by method. Detection rates were measured at 800px, 1200px, and 1500px. OpenCV was tried as well — it failed on Version 40 entirely. jsQR stabilized at 1500px, where each module reached about 5.1 pixels. We also discovered that PDF link annotations could extract URLs directly without scanning QR codes at all, achieving 100% accuracy.
At the time, the approach was a two-tier strategy: annotation extraction as the primary method, jsQR at 1500px as fallback. The approach itself became obsolete with the shift from PDF to ZIP, but the lesson remained: QR generation and readability must be guaranteed by design, not by assumption. This later led to building automated verification into the server-side pipeline — scanning the generated PDFs' QR codes and cross-checking them against the source data. But the next constraint was already waiting — in bulk mode, the URLs packed into QR codes exceeded Version 40's capacity. Every time one constraint was resolved, the next one appeared.
Auto-merge Race Conditions
TokiQR's repositories have auto-merge workflows: once CI passes, the PR merges immediately. Convenient, but when pushing additional commits to an existing branch, the merge sometimes completed before the push arrived. Changes were left stranded.
Retreat or Push Through
PDFs ballooning in size, GitHub Actions failing, QR codes unreadable. As constraints piled up before my eyes, what crossed my mind were options for retreat.
Maybe bulk mode should be dropped. Maybe opening deposit to the general public was unrealistic from the start. Maybe it would be better to switch to duplicating the system per customer with separate accounts. Or perhaps a "legend" slot in the main newsletter — selecting just one person every six months — would dramatically reduce pipeline load. Each option looked rational for a moment.
But every one of them meant cutting either functionality or value. Dropping bulk mode would make it impossible to deposit long-form audio. Closing off public access would narrow the meaning of a thousand-year preservation service. Per-customer duplication would create operational costs that grow linearly forever. The legend slot could create an air of exclusivity, but it would mean abandoning the core premise that anyone can preserve their voice. Instead of surrendering functionality in the face of constraints, I decided to find routes that bypass the constraints while keeping both functionality and value intact. Considering retreat, paradoxically, is what cut off the retreat. By articulating exactly what would be lost with each fallback option, the only remaining path was forward.
Breakthroughs
Consolidation into newsletter-master
I abandoned the per-customer repository approach. All series' ZIPs and PDFs were consolidated into a single repository called newsletter-master. GitHub Actions handles the build; GitHub Pages handles delivery. Series listing pages and individual pages are auto-generated by a Python script.
This eliminated the bandwidth problem. Push a ZIP file, and the workflow generates PDFs, updates the index pages, and deploys to GitHub Pages. The entire process completes server-side.
The Shift from PDF to ZIP
I stopped using PDF as the intermediate format and pivoted to ZIP. Audio data, manifest.json, and metadata are bundled into a single ZIP that becomes the pipeline's primary source. PDFs are generated server-side as derivatives of the ZIP.
Three things happened simultaneously with this shift. Moving PDF generation server-side allowed jsPDF and its font files (2.7MB) to be removed from the client, and the generated PDFs shrank dramatically. ZIPs serve directly as playback data sources, so adding a ?zip= parameter to play.html was all it took to enable newsletter playback. And with ZIPs as the primary source, repository bloat was reduced as well.
Queue-based Asynchronous Processing
NDL submissions were working. As long as one person submitted one item at a time. But the moment I asked "what happens when a hundred people do this simultaneously?" the constraints surfaced. A design that hits the GitHub API for PDF generation and git push with every submission in real time would run into rate limits (5,000 requests per hour, authenticated). The moment usage scaled, it would choke. Something that works is not the same as something that scales.
I switched to placing ZIPs in a queue/ directory and processing them on a five-minute timer. Failed submissions are retried automatically. Users can move on to their next task without waiting for results. By letting go of real-time processing, the concurrency constraint dissolved.
The One-Branch-One-Commit Discipline
The auto-merge race condition was caught by a discipline established from the start of development. Each branch gets exactly one commit. Additional changes are made on a new branch only after confirming the previous PR has merged. The problem of pushes arriving after auto-merge completes was encountered from the very first days of writing code. It may look like more steps, but the risk of stranded changes drops to zero.
What Sixty PRs Taught Me
Looking at the breakdown of the roughly sixty pull requests from those two days, a pattern emerges.
- qr (20 PRs): NDL pipeline integration, ZIP infrastructure, bulk QR fixes, UI improvements
- newsletter-master (25 PRs): Repository creation from scratch through UI construction and workflow setup
- lp (18 PRs): Pipeline migration, queue-based async processing, newsletter page overhaul
Few of these qualify as new features. Most were re-plumbing of existing systems. Rerouting data flows, bypassing bottlenecks, splitting thick pipes into thinner ones. The change visible to users was minimal, but the flow underneath was entirely different.
Constraints were not enemies of design — they were inputs to it.
What Happens at the Border of Development and Operations
In the development phase, the goal is to build something that works. In the operations phase, the goal is to keep it working. This difference manifests not in how code is written, but in how pipelines are designed.
The repositories were split into three from the development stage. The client (qr), business logic (lp), and server-side builds (newsletter-master). The structure was the same, but in operations the role of each repository became sharply defined. Each is deployed independently and changes at its own pace. What was merely "kept separate" during development became "needs to be separate" in operations.
Bandwidth limits demanded repository consolidation. Binary bloat demanded a format change. Dependency weight demanded client-server separation. The separation made during development was not a coincidence — the constraints confirmed it after the fact.
Closing the Circuit with Verification
The pipeline was working. ZIPs in, PDFs out, deployed to GitHub Pages. But one piece was missing. There was no mechanism to verify that the QR codes in the generated PDFs could actually be read back from the page.
TokiQR's thousand-year preservation is meaningless if QR codes cannot be scanned. Trusting that the generation side "embedded them correctly" is not enough. The PDFs need to be rendered, QR codes scanned, and the results compared against the original URLs.
I added verify-qr.py. Each PDF page is rendered at 300 dpi, QR codes are detected with pyzbar, and the results are cross-checked against the URL list in the ZIP's manifest.json. If even one URL is missing, the build fails and deployment is blocked. I ran it against all six existing PDFs — all passed.
There was one more gap. When verification failed and deployment was halted, there was no way to notice. Success notifications become noise as volume grows. Only failures need to be reported. I added a step that automatically creates a GitHub Issue when a build fails. Silent on success, alert on failure.
How to Isolate Problems in the Operations Phase
Looking back at this renovation, a repeating pattern emerges in how problems were found and fixed.
First, question what appears to be working. It worked when one person submitted one item at a time. But what happens when a hundred do it simultaneously? The PDF was generated, but can the QR codes actually be read? When a build stops, will anyone notice? Doubting the working state is the starting point.
Next, locate the constraint. Where exactly is the problem? Is it the GitHub API rate limit, the QR resolution, or the absence of notification? Do not stop at "it's not working" — isolate which joint in the pipeline is clogged.
Then, redesign the pipeline around the constraint. Replace real-time processing with a queue. Swap client-side PDF parsing for server-side automated verification. Remove success notifications and keep only failure alerts. The goal is not to eliminate constraints but to find routes where they do not apply.
Finally, embed the improvement into the system. Not a manual check someone must remember, but automated verification that runs as part of the pipeline. Only when improvement works without human memory does it become permanent.
Established Frameworks as Guide Lines
Running this cycle on your own is not hard. But knowing established frameworks reduces blind spots at each stage. Here are several concepts adjacent to this renovation.
DFD (Data Flow Diagrams) helps with the "locate the constraint" stage. Drawing the flow — ZIP → PDF generation → QR verification → deployment — makes it visually clear which joint is clogged. In this case, the junction with the GitHub API was the bottleneck. The basic act of diagramming data flows accelerates constraint isolation.
TOC (Theory of Constraints) is effective at the "redesign" stage. Goldratt's theory teaches that the throughput of an entire system is determined by its slowest process. Instead of optimizing everything, identify the single bottleneck and address only that. By routing around the API rate limit with a queue, the entire pipeline started flowing.
Pipes and Filters is the architectural pattern that describes this design itself. Each stage is an independent processing unit that receives input, transforms it, and passes it on. The reason verify-qr.py could be inserted after PDF generation was that the pipeline was loosely coupled.
PDCA (Plan-Do-Check-Act) overlaps with the "embed in the system" stage. But the lesson here was that Check should be done by the pipeline, not by people. Without automating verification, the cycle eventually stops.
Frameworks do not produce answers on their own. But they draw guide lines for "what to think about next." When facing a constraint, the more concepts you have at hand, the faster you see candidate routes.
Alternatives at Each Layer
Once frameworks draw the guide lines, the next step is to lay out alternatives at each layer. Here are the options actually considered during this renovation, and why each was chosen.
Data format layer — PDF alone / ZIP+PDF / ZIP alone. PDFs cannot be diffed as binaries in git, bloating the repository. ZIP+PDF makes PDFs a derivative of ZIPs, cutting client dependencies and reducing file sizes. ZIP was adopted as the primary source, with PDFs generated server-side.
Repository structure layer — Per-customer repository / consolidated single repository. The per-customer model was vulnerable to API rate limits, and linear growth in repository count became an operational burden. Consolidation into newsletter-master made API costs constant.
Submission timing layer — Real-time synchronous / batch queue / event-driven. Real-time choked on API rate limits. Event-driven required additional infrastructure such as Cloudflare's durable queues. A file-based queue with a five-minute timer required zero additional dependencies and ran entirely on GitHub Actions cron.
QR verification layer — Client-side jsQR / server-side OpenCV / server-side pyzbar. jsQR runs in the browser but is highly resolution-dependent. OpenCV failed on Version 40. pyzbar, a C-based ZBar wrapper, achieved stable detection rates when combined with 300 dpi rendering.
Notification layer — Notify on both success and failure / failure only / no notification. As volume grows, success notifications become noise. Failure-only notification via automatic GitHub Issue creation was chosen, with direct links to logs for immediate investigation.
The act of laying out alternatives is itself an exercise in articulating design decisions. When you can state explicitly "why this was chosen," your future self — or anyone else — can re-evaluate the choice when constraints change.
DFD, TOC, Pipes and Filters, PDCA. These concepts are not specific to this pipeline. They work equally well on a system you built from scratch and one you are seeing for the first time. Draw the data flow, locate the bottleneck, find the loosely coupled joints, automate verification. With these mental models already in hand, you can arrive at solutions clearly and boldly — no matter what environment you are dropped into.
What AI Can Do, and What Remains
This renovation was carried out in collaboration with AI. Code generation for verify-qr.py, GitHub Actions workflow modifications, simultaneous drafting of this essay in Japanese and English. AI is fast at applying established patterns and implementing code based on known frameworks.
When most people use AI, they aim to produce a finished product in a single pass — refining prompts, trying to get the final version from one exchange. What this renovation did was fundamentally different. I used AI to generate rapid drafts, identified constraints with my own eyes, and redrew the structure from scratch — cycling through this process repeatedly. AI's output is raw material; I sculpt it with my own questions. And those questions are grounded in constraints earned through actual product operation.
At the same time, throughout this prototyping process, AI repeatedly suggested increasing external dependencies. Store it in Google Drive for easier processing. Manage it in a spreadsheet for simpler coding. Each suggestion looked reasonable on its own. But for a service built on the premise of thousand-year preservation, adding dependencies on external services contradicts the core design philosophy. I reviewed each suggestion carefully and corrected course. AI proposes efficient solutions to the problem at hand, but it does not ask whether those solutions align with the design philosophy at the heart of the product.
But the question "what happens when a hundred people do this simultaneously?" did not come from AI. The judgment that "success notifications become noise as volume grows" was born from a human imagining the reality of operations. AI can apply frameworks. But deciding which framework, at which moment, against which problem — that is the job of whoever poses the question.
I believe this structure will not change no matter how far AI evolves. The power to solve can be amplified endlessly. But "what to ask" is not subject to amplification. The quality of the questions that surface constraints determines the quality of the solutions. The value of knowing frameworks lies in becoming able to hand AI the right questions.
More people than ever are proficient at using AI as a tool. But very few can clearly articulate what their own role should be in collaborating with AI — and demonstrate that answer through practice. Without noticing the absence of questions, they accept AI's output as the finished product. This renovation record serves as a living proof of that distinction.
This structure holds whether you are an individual developer or an enterprise. In fact, the larger the organization, the more severe the damage when the right questions are absent. AI has accelerated the generation of proposals, making it easier than ever to adopt initiatives that look compelling but lack grounded expertise. Open the lid and you find no coherence, no fit with on-the-ground realities, and foundations maintained for years begin to buckle — extended service outages are no longer rare. The same structure produces security holes: AI-generated code integrated without architectural deliberation may look correct in isolation, but without questioning the broader context, it becomes a vulnerability. And the larger the scale, the greater the consequences — personal data breaches and cyberattacks become exponentially more damaging. Being able to build fast and being able to ask the right questions are different capabilities.
To be clear, this is not a dismissal of the strengths that enterprises bring. Large organizations have depth of resources, redundancy, and accumulated governance that individuals cannot replicate. The point is that leveraging those strengths requires knowing what to ask. I work independently at times, and at other times collaborate with enterprises through partnerships. Regardless of scale, the structure remains the same: the quality of the questions determines the quality of the design.
This renovation converges here. The phase of making fundamental changes to the code is over; the system has transitioned into operations. Where and when future changes might arise has been mapped out separately in a migration roadmap. The reason I am not touching the code now is not oversight — it is a deliberate decision, made after identifying the points of change, to leave things as they are. Operational maturity is not about perpetual modification. It is the ability to discern when and where to intervene, and to wait until that moment arrives.
In an era where AI can amplify the power to solve without limit, spending time alone no longer creates value. A single line of questioning can reshape an entire structure, while a hundred hours of labor may change nothing. Sharpen the questions, apply leverage, build a state where you can wait — and then build more of them. That is the shape of work this renovation record demonstrates.
And TokiStorage as a product is itself the embodiment of this structure. The moment a QR code is printed on paper, it needs no server, no subscription, no electricity. The decoder is just 3 MB; the data is distributed across URLs on the QR surface; the server side does not scale with the number of records. Every new customer creates one more state that can wait. The burn rate of the business is designed to be zero. The pipeline, the product, the business structure — all of it stands as a state of waiting.
From individual development to enterprise. From designing for permanence to establishing systems and operations.
We offer perspectives that AI alone cannot reach — through dialogue.
What constraints taught me was not what to give up, but where the flow could pass through.