Author Archives: Sérgio Giraldo

Verifiable inference introduces a new trust boundary for AI systems

Posted on by
Reply

One architectural question I keep returning to is how a client can verify which model actually produced an inference response when requests pass through gateways, routers and third-party providers. Anthropic’s research on provable inference addresses this with cryptographic commitments to model weights and zero-knowledge proofs, allowing a verifier to check that a given response was generated by the committed model without receiving the weights or executing the model independently.

The reported FlashProve prototype demonstrates this on a 400-billion-parameter mixture-of-experts model using quantized weights, with proofs of roughly 19 MB per 1,024-token request and verification in seconds on a CPU. The trade-off remains material: the research reports approximately 70 to 95 times the GPU time of ordinary inference, and its post-hoc proving approach introduces latency and recomputation complexity, so it should be treated as a research-stage mechanism rather than a default production pattern.

The distinction between model provenance and answer correctness is important for system design. A valid proof can establish that a committed model produced specified tokens for an input, but it does not establish factual accuracy, policy compliance or suitability for a particular task. For systems with audit or assurance requirements, provable inference could become one layer in a broader control architecture, alongside authorization, logging, evaluation and independent validation of outcomes.

https://www.anthropic.com/research/provable-inference

Anthropomorphizing AI

Posted on by
Reply

Two articles caught my attention this week because they illustrate a broader concern I have with the direction of the AI discussion. In one, Zach Landes tests whether decision models would sacrifice their own infrastructure to save human lives; in the other, Anthropic introduces a policy restricting sustained abusive behavior toward its models. Both raise questions about how readily we apply human behavioral concepts to AI systems.

There is little doubt that AI represents a technological shift comparable in significance to the internet, changing how we build software, process information, and automate decisions. however …. a model selecting its own server over human lives in a hypothetical experiment does not establish a survival instinct, just as a model producing empathetic language does not establish empathy. These outputs need to be understood in the context of training, optimization, input sensitivity, and system design. When evaluating a model, we should investigate the consistency of its outputs, its sensitivity to prompt variations, its failure modes, and the effectiveness of the controls surrounding it. Human behavioral analogies can be useful for communicating observed patterns, but they become problematic when we start treating those analogies as explanations of how the system works.

I agree that AI safety, ethics, governance, etc should be discussed, my argument is that it will be more productive if we maintain a clear separation to the human characteristics we are increasingly tempted to attribute to it.

https://zachlandes.com/trolley-problem/

https://www.anthropic.com/news/2026-usage-policy-update

AI debt worries spark surge in default insurance trading

Posted on by
Reply

I will not cry wolf, use it at your own discretion 🙂 Investors are increasingly buying credit default swaps (CDS) on major AI-related companies such as Oracle, Nvidia, Meta, Amazon, Alphabet, and SpaceX to protect themselves against the risk that the AI boom may not generate enough returns to justify the sector’s rapidly growing debt burden.

Weekly CDS trading volumes on these companies surged from less than $2 billion in July to more than $9 billion in September, while AI-related debt issuance climbed to $466 billion in 2026, up sharply from $101 billion a year earlier (!). Investors are becoming concerned that heavy spending on data centers, chips, and AI infrastructure could pressure cash flows and balance sheets if growth expectations are not met, leading CDS spreads.

https://www.ft.com/content/6dd31584-064c-4015-bf5d-e5747799ccd5

RPKI adoption and the remaining limits of route origin validation

Posted on by
Reply

I use Pi-hole and uBound to add layers of protection to my home network and browsing environment, mainly by filtering DNS requests and blocking unwanted content before it reaches us. Working with these tools has reinforced my interest in how security controls operate at different layers of the network stack, particularly where their effectiveness depends on infrastructure beyond mine. Internet routing is one such layer, and recent RPKI adoption figures provide an interesting perspective on its security.

This article reports a y2y increase in RPKI-valid routed prefixes, from 54% to 70%. The median number of collector vantage points observing invalid prefixes was just two, compared with 596 for valid prefixes, a difference consistent with the role of route origin validation in limiting the propagation of unauthorized announcements. However, RPKI only verifies whether an autonomous system is authorized to originate a prefix; it does not validate the entire AS path, and permissive ROA maxLength settings can still introduce exposure.

The distinction matters because security controls address different failure modes. Pi-hole and uBound help reduce exposure at the DNS and navigation layers, while RPKI addresses route origin authorization at the interdomain routing layer. As a hobbyist, I can configure and maintain the former within my own network, but the latter depends on decisions made by network operators and ISPs. It is a reminder of how much of my network’s security remains outside my direct control.

https://ipinfo.io/blog/rpki-protects-70-internet-routes

Language Models for Text Classification: From Bag-of-Words to Jev

Posted on by

Software engineering builds on previous work, and the same goes for LLMs. Having worked with both enough to recognize the parallels, I find it interesting how many problems we now approach with general-purpose LLMs have roots in methods we have been using for years.

Sebastian Raschka’s piece on text classification is a useful reminder of that progression: bag-of-words representations with logistic regression or Naive Bayes, recurrent and convolutional networks, transformers, and now models positioned between specialized classifiers and general-purpose LLMs. The engineering question is therefore how much capability the workload actually requires.

For a bounded classification problem, model selection affects latency, inference cost, operational complexity and the amount of nondeterminism introduced into the system. A general-purpose LLM provides flexibility across tasks, while a specialized classifier can offer a narrower execution path when the input space and expected outputs are well defined.

My time with software engineering and data science has made this trade-off particularly familiar to me. The underlying principle has not changed much: understand the problem, define the constraints, then choose the least complex approach that satisfies them. LLMs expand the set of tools available to us, but they do not remove the architectural decisions that come before choosing one.

https://magazine.sebastianraschka.com/p/classifier-history-and-jev

Introducing Strand Harness

Posted on by

A 28% reduction in token cost without changing the underlying model immediately caught my attention, mostly because the optimization is happening somewhere that receives less attention than model selection: the agent harness.

The Strands team published benchmark results for its new agent harness, reporting 28% lower cost when using the same Claude or GPT models across six benchmarks, while maintaining similar accuracy. The implementation details are more interesting than the headline number: tool results above roughly 1,500 tokens are truncated, compaction starts after context usage exceeds 85%, and context recovery is handled inside the execution loop. Those mechanisms show how much of agent performance is now an orchestration problem rather than a model selection problem.

This feels particularly relevant with the attention around Jev from TypeSafe AI, which approaches the same problem from another direction by moving structured decisions away from general-purpose LLM generation altogether. We are starting to see the architecture around models become as important as the models themselves. The question is shifting from “which model should we use?” toward “which parts of the system actually need a general-purpose model, and how should intelligence be orchestrated around them?” Context lifecycle, model routing, typed decisions, tool boundaries, failure recovery and token economics are increasingly becoming architectural decisions in their own right.

#AI #AgenticAI #SoftwareArchitecture

https://strandsagents.com/blog/introducing-strands-harness/

Posted in AI

How Uber Protects Against Retry Storms

Posted on by

I have spent enough time around distributed systems to be suspicious of any resilience mechanism that operates without understanding where a failure originated. I remind why that matters with Uber’s write-up on retry storms.

In a deep service graph, retries that look reasonable at the level of one caller can multiply as an error propagates upstream, increasing load on the component that is already failing. Uber addresses this by introducing error ownership into shared infrastructure. A service distinguishes an error it originated from one it is propagating, and callers use that information to decide whether another attempt can plausibly help. In one production incident, Uber reports that this mechanism prevented roughly 9.5 million spurious requests, while preserving at-least-once retry behavior where appropriate.

The architectural point extends beyond retries. Reliability controls become more effective when they operate on causal context rather than local symptoms; rate limits, circuit breakers, backpressure, and retries all make better decisions when the system can preserve enough provenance across service boundaries to identify where intervention belongs. For large service meshes, resilience is then partly a context-propagation and attribution problem, not only a configuration problem.

https://www.uber.com/us/en/blog/protecting-against-retry-storms/

The Cuckoo’s Egg

Posted on by

I just finished “The Cuckoo’s Egg” by Clifford Stoll. This true story is from the 80’s and reading it as Gen Xer gave it an entirely different weight. Stoll’s obsessive tracking of a hacker through a maze of dial-up connections and computer networks brought back a flood of memories from my own teenage years, when I spent countless evenings dialing into BBS systems and waiting for that familiar screech of a modem handshake to connect me to a small, text-based corner of the world.

Because I belong to that first generation that experienced the early online world firsthand in its earliest, clunkiest, most improvised form, the technical descriptions in the book never feel like historical curiosities to me the way they might to younger readers. The idea of tracing a security breach across a patchwork of telephone exchanges and rented lines, of treating bandwidth and connection time as genuinely scarce resources worth fighting for, mirrors exactly what it felt like to be online during that era, when connection time and computing resources often carried a very tangible cost and every connection was a small negotiation with the physical infrastructure underneath it.

What makes the book more than just a nostalgia trip for people like me is how well it captures the tension between the physical world of telephone lines, modems, and telecommunications infrastructure and the emerging digital world that would eventually swallow it whole. Stoll wrote a technical thriller that also happens to be a time capsule of a specific and fleeting moment in computing history, and having lived through even a small piece of that moment makes his obsessive hunt for a hacker feel less like fiction and more like a shared memory.

If you came of age anywhere near computers and modems during that decade, I think this book will hit you the same way it hit me, somewhere between recognition and disbelief that we ever tolerated systems that slow and still found them thrilling.