#137foundry

[137-foundry]

What We Learned Auditing Loading States Across a Dozen Client Dashboards

Over the last year we've done loading state audits as part of maybe a dozen different engagements, usually tacked onto a broader front-end review rather than requested on their own. Nobody asks for a "loading state audit" specifically. But the pattern of what we find is consistent enough now that it's worth writing down.

How These Audits Usually Start

A client brings us in for something else entirely, a performance review, an accessibility pass, a general front-end health check, and loading states come up as a side finding almost every time, not because we go looking for them specifically but because they're one of the first things a systematic review surfaces. Once you're clicking through every major flow in an application looking for issues, loading states are impossible to miss, because you hit one on nearly every screen.

We started keeping informal notes after the third or fourth engagement where the same three issues showed up independently, on completely unrelated codebases, built by different teams, in different frameworks. That kind of consistency across unrelated projects usually means the root cause isn't a specific team's mistake. It's a gap in how loading states get taught and reviewed industry-wide.

The Same Three Problems, Almost Every Time

The first is a progress bar that isn't measuring anything real. Somewhere in the codebase, a developer decided a spinner "felt incomplete" and added a bar that fills based on elapsed time rather than actual completion. It looks more informative than a spinner and is actually less honest, because it's implying a measurement that doesn't exist. Users notice this faster than teams expect, usually within the first few uses, and it quietly erodes trust in every other progress indicator in the product afterward.

The second is a loading state with no timeout. If a request stalls, silently, without ever resolving or erroring, the spinner just keeps spinning. We've found this in production more times than we'd like to admit. The fix is small (a timeout that swaps to a retry state after 8 to 15 seconds) but it's almost never there by default.

The third, and the one that surprises people most, is loading states that are completely invisible to screen reader users. A spinner is a purely visual signal. Without an aria-live region announcing when the load completes, a screen reader user gets total silence during the wait and no explicit confirmation that content actually appeared. MDN's documentation on ARIA live regions is the reference we point every team to for this, and WebAIM has good practical examples of applying it to real interfaces, not just spec text.

Why This Keeps Happening

Loading states get built under time pressure, usually as one of the last things wired up before a feature ships, and they get built once and never revisited unless something breaks visibly. Nobody files a bug report that says "your loading state doesn't announce to screen readers," because the people affected by that gap usually can't tell whether it's a bug or expected behavior. It just quietly fails for a subset of users who don't have a clear channel to report it.

The fix isn't more design time. It's a short checklist applied consistently: does the timing match what the system actually knows, is there a timeout, is the completion announced, does the skeleton shape match the real content closely enough to avoid a layout jump. Running through those four questions on any loading state we touch catches the majority of what we find in audits, and we've started handing this same short checklist to client teams directly so they can run it themselves between our engagements rather than waiting for the next audit to surface the same recurring issues again.

What Surprised Us Most

Going in, we expected the accessibility gap to be the rarest finding, something only teams with an existing accessibility practice would have thought about at all. It turned out to be almost universal in the other direction: across the dozen engagements, only one had any aria-live handling around loading state transitions, and that one had it because a specific accessibility audit had flagged it months earlier as a standalone issue.

That tells us the gap isn't really about team skill or care. It's that loading state accessibility sits at the intersection of two things nobody explicitly owns: it's not quite a design responsibility (designers think about the visual shimmer, not the screen reader announcement) and it's not quite treated as a core engineering responsibility either (engineers wire up the data fetching and consider the visual state "done" once it renders correctly). It falls into the gap between those two ownership boundaries more often than almost any other accessibility issue we find.

The fix we recommend is organizational as much as technical: bake the aria-live pattern into whatever shared component or hook handles loading states in the codebase, so it's automatic for every feature built on top of it rather than something each team has to remember to add individually.

The One That Was Hardest to Explain to Stakeholders

The fake progress bar finding is consistently the hardest one to get buy-in on fixing, because it usually tests well in isolation. Show a stakeholder a mockup with a smooth, steadily filling progress bar next to a plain spinner, and the progress bar reads as more polished, more informative, more "finished" as a piece of design. The problem only shows up over repeated real use, once a user has seen the bar jump or stall inconsistently enough times to stop trusting it, and that kind of trust erosion doesn't show up in a single mockup review.

The argument that eventually works is usually a concrete example from the client's own product: pulling up a session recording where a real user watched a fake progress bar behave inconsistently, then comparing it to how a plain, honest spinner with a status label would have handled the same wait. Seeing the actual user reaction, hesitation, repeated clicking, abandoning the flow, tends to land better than an abstract argument about honesty in interface design. Once a team has watched that footage once, the fake progress bar tends to get fixed without much further debate, and it usually becomes the example that gets pulled up the next time a similar shortcut is proposed elsewhere in the product. We've started keeping a short highlight reel of these moments across engagements, with permission, specifically because they do more to change a team's default habits than another slide of abstract UX guidance ever does.

"None of the loading state problems we find are hard to fix individually. What's hard is that they're spread across dozens of components, built by different people, at different times, with no shared checklist. That's a process gap more than a skill gap." - Dennis Traina, founder of 137Foundry

We put the full breakdown of spinners versus skeleton screens versus real progress bars, and when each one is actually the right call, into a longer guide here: designing loading states that don't feel like lying to users. If you want an outside pair of eyes on your own product's loading patterns, that's a normal part of the front-end work we take on at https://137foundry.com.

[137-foundry]

Why We Replaced Spinners With Skeleton Screens on a Recent Client Build

A few months ago we shipped a dashboard rebuild for a client whose users were filing tickets that all said some version of the same thing: "the app feels slower than it used to." The odd part was that our own performance instrumentation said the opposite. Median load time had actually dropped by about 15% compared to the previous version.

We spent a week convinced the metrics were lying before we figured out the metrics were fine. The interface was lying.

Where We Started Looking

The first instinct on a project like this is to distrust the monitoring, so we spent the first couple of days re-verifying our own numbers before we trusted them. Real user monitoring, synthetic tests from three different regions, server-side timing logs, all of it agreed: things had gotten faster, not slower, on every dimension we were tracking.

That's a genuinely uncomfortable place to be as an engineering team, because "the users are wrong" is rarely the right conclusion, and "our metrics are lying" was demonstrably false once we'd checked it three different ways. The only remaining explanation was that we were measuring the wrong thing entirely, that the users' experience of speed and our instrumentation of speed had quietly diverged.

What Was Actually Happening

The old dashboard had a full-page spinner that appeared the instant you navigated to it and stayed until every widget on the page had finished loading, sometimes six or seven separate data fetches. The new version was objectively faster in aggregate, but it rendered its first content later relative to when the spinner appeared, because we'd restructured the data fetching to be more efficient in total but less front-loaded.

Users don't experience total time. They experience the interval between "I did something" and "I saw something happen." We'd made the first number better and the second number worse, and the second number was the one people actually felt.

The Fix Wasn't a Backend Change

We replaced the full-page spinner with a skeleton layout matching the dashboard's actual grid: placeholder cards in the exact positions the real widgets would occupy, each one resolving to real content independently as its data arrived, instead of waiting for all seven fetches to finish together.

This meant the page started looking "alive" within a few hundred milliseconds, even though the slowest widget on the page still took the same amount of time it always had. Nothing about the backend changed. What changed was that the interface stopped hiding its own progress from the user.

Nielsen Norman Group's research on response time thresholds was a useful gut check while we made this call. The window between one and ten seconds, where most of these individual widget loads fell, is exactly the range where interface design determines whether a wait feels managed or ignored.

What We'd Do Differently Next Time

The one thing we underestimated was how closely the skeleton shapes needed to match the real content. Our first pass used generic rectangular placeholders that didn't account for the fact that one widget rendered a chart and another rendered a short text summary. When the real content swapped in, the layout shifted more than we expected, which reintroduced a smaller version of the same "something feels off" complaint we were trying to fix.

The second pass matched the skeleton shapes to the actual rendered dimensions of each widget type, checked against web.dev's Cumulative Layout Shift guidance, and that fixed it. It's a detail that's easy to skip under deadline pressure and expensive to skip in practice.

How We Verified It Actually Worked

We didn't just ship the skeleton version and assume the problem was solved. We watched a handful of session recordings from before and after the change, specifically looking at what users did in the first two seconds after navigating to the dashboard. Before the change, a meaningful number of sessions showed a click on the browser refresh button within the first second, a strong signal that the blank spinner state read as "nothing is happening" rather than "loading."

After the change, that refresh-click pattern dropped close to zero. Nobody was refreshing a page that was visibly already rendering content, even partial, placeholder content. That single behavioral signal told us more about whether the fix worked than any aggregate timing metric did, because it captured exactly the thing we were trying to fix: the moment of uncertainty where a user decides whether to trust that something is happening.

Ticket volume mentioning "slow" or "frozen" dropped by roughly half over the following month, without a single additional backend change. The lesson generalized past this one project: for any product where users describe something as feeling slow despite metrics saying otherwise, the fix is very often in how the wait is communicated, not in the wait itself.

The Part That Took Longer Than Expected

Building the skeleton layout itself took about two days. Getting the skeleton shapes to actually match the real widget dimensions took closer to a week, because it meant going back through every widget type in the dashboard, measuring its real rendered dimensions across a range of realistic data (a chart widget with three data series looks different from one with twelve), and building a matching placeholder for each variant rather than one generic block reused everywhere.

That ratio surprised us. We'd budgeted the project assuming the skeleton implementation itself was the hard part. It turned out the shimmer animation and placeholder markup were trivial. The actual work was in the audit: figuring out what the real content shapes looked like across enough realistic scenarios that the skeleton didn't introduce its own layout shift once real data arrived. If you're estimating a similar project, budget the audit time separately from the implementation time. They're genuinely different kinds of work, and treating them as one estimate is how this kind of project ends up running over its original timeline without anyone being able to point to a specific reason why.

"The lesson that keeps repeating on projects like this is that perceived speed and actual speed are two different engineering problems, and most teams only budget time for one of them." - Dennis Traina, founder of 137Foundry

A few weeks after this project wrapped, a second client asked us to look at a strikingly similar problem on a completely different codebase, a mobile-first booking flow where the reported complaint was "the app hangs when you tap search." Same root cause, a blank screen during a genuinely brief wait, same fix, a skeleton preview of the results list shape. It's become one of the more reliable diagnostic shortcuts we reach for now: when the metrics and the complaints disagree, check what the user actually sees during the wait before assuming either one is wrong.

We wrote up the fuller framework, when to use a skeleton versus a spinner versus a real progress bar, in a longer guide if you want the details beyond this one project: designing loading states that don't feel like lying to users. And if your product has this exact "the metrics say fast but users say slow" gap, it's usually a loading state problem before it's anything else, which is the kind of front-end work we do a lot of at 137Foundry.

[137-foundry]

How We Approach Data Integration Replay Tooling at 137Foundry

A perspective post, written from the inside of how we actually build these systems for clients, on what makes the difference between a replay tool we feel good about shipping and a "run this script and hope" approach that the client will inherit and fear.

The short version: we treat replay capability as a first-class feature of any integration we build, not a recovery afterthought. The reasoning has been the same across enough client engagements that we now bake it into the integration template from day one.

Photo by Sergei Starostin on Pexels

The pattern we keep landing on

Across roughly twenty data integration projects in the last few years, the replay design we trust is always the same shape:

A durable per-event log on our side of the integration, indexed by upstream event ID, stamped with ingest time and processing outcome.

A check-then-act block in the processing code that looks up the existing downstream record by event ID before deciding to create or update.

A separate writer layer that the processing code calls into for all side-effecting work, so the same code path can run in dry-run mode or live mode.

A replay command that takes mandatory time-window arguments, defaults to a conservative rate limit, and supports a dry-run mode out of the box.

A dashboard view that shows the event log's recent ingest counts, processing outcomes, and any backlog, so on-call can see at a glance whether the integration is healthy.

The reason we keep landing on this shape is that every client engagement we have done has eventually faced the same situation: an upstream outage, a few hours of missed events, and a need to recover without producing duplicates. The teams that had this pattern in place recovered cleanly. The teams that did not had a follow-up incident that was usually worse than the original outage. The pattern is not novel; it is just the answer to the question, and we have stopped pretending each client's situation requires a fresh design.

What we have learned to push back on

A few specific client requests come up often, and we have learned to push back on them politely but firmly.

"Can we skip the per-event log and just rely on the upstream system's buffer?" We almost always say no. The upstream system's retention is theirs; they will change it without telling us; the first time we find out is during an outage. The per-event log is ours and we control its retention. The cost is modest. The value during recovery is the entire point.

"Can we make the replay default to running everything since the outage started?" Also no. The right replay scope is "the specific events that did not succeed," not "everything in the time window." The two are different sets, and replaying the second produces duplicates from events that did succeed during partial-availability periods. We bake the right scoping into the replay tool's interface so the wrong scoping is not an easy mistake.

"Can we skip the dry-run mode for now and add it later?" This is the one we feel most strongly about. A replay tool without a dry-run mode is operationally dangerous, and the version of "later" where the team adds the dry-run mode is almost always "after the first incident where we wished we had it." The marginal cost of building the writer layer with both modes from the start is small. The cost of retrofitting it later is large.

"Replay tooling is one of those features that looks like overkill until the first time you need it, and then it looks like the cheapest thing you ever built. The pattern is well understood; the discipline is in actually building it before you need it instead of after." - Dennis Traina, founder of 137Foundry

The retrofitting cost is what motivates us to push for the full pattern from day one, even on integrations that are "simple enough not to need it." The integrations that turned out not to need a replay tool cost us a small amount of work to build one. The integrations that turned out to need it would have cost the client a multi-day incident if we had not.

Photo by Atlantic Ambience on Pexels

What "good" looks like from the operator's seat

We measure the replay capability against a specific scenario: an on-call engineer, alone, at 2 AM, after a multi-hour upstream outage. Can they:

See on the dashboard that the integration was degraded between time A and time B.

Query the event log for the specific events in that window that did not succeed.

Run a dry-run replay scoped to those events and read the expected outcome counts.

Run a live replay scoped to the same events with the default rate limit.

Confirm via the dashboard that the gap closed.

If any of those steps requires a Slack thread with a senior engineer, ad-hoc SQL against the production database, or hand-editing a script during the incident, the capability is not done. We test against this scenario explicitly before we consider a replay tool shipped. The "drill" is a real exercise we run in staging, with the on-call engineer who will actually be carrying the pager.

The drill is uncomfortable the first time. The second time it is just a habit. By the third one, the engineer has the muscle memory to do it cold at 2 AM, and that is the entire point.

The architectural decisions that compound

A few specific design choices pay off across every integration we have built. They are not novel either; we are not the first people to land on them. But they compound across the lifetime of the integration in ways that are easy to underweight at design time.

The first is the writer layer. Pulling all side-effecting calls behind a writer interface costs maybe a day of design work at the start of the project. Across the lifetime of the integration, it makes dry-run mode possible, makes testing radically easier, makes the next migration to a different downstream system tractable, and prevents an entire class of "we forgot to honor the dry-run flag in this one path" bugs.

The second is the per-event log. Adding a Postgres table to the integration from day one costs an hour. Across the lifetime, it makes replay scoping correct, makes the "where did this event go" debugging question answerable in five minutes instead of an afternoon, and makes the integration legible to operations engineers who did not write it.

The third is the rate-limited replay default. Setting a conservative default rate on the replay tool costs zero effort. Across the lifetime, it prevents the "the replay finished, but now the downstream system fell over from the burst" follow-up incident exactly once for each integration that runs into it.

None of these are large decisions individually. Together they are the difference between an integration that the team feels comfortable operating and one that the team is afraid of. The discipline of making all three decisions on day one, not after the first incident, is what we have ended up codifying.

For the full architectural walkthrough of how we build the replay tooling itself, the 137Foundry article on data integration replay covers the event log schema, the idempotency key design, the dry-run mode, and the operational controls. The 137Foundry data integration service page has the surrounding context for how we approach integration projects from initial design through production. For the broader set of reliability patterns we lean on across client work, the 137Foundry articles cover related topics like schema drift detection, integration error queues, and idempotency for retry-prone webhooks. The underlying technical concepts are well-grounded in widely cited references like the Wikipedia article on idempotence and the Apache Kafka documentation, which most teams already know.

[137-foundry]

Five Mistakes We Have Made Building Integration Replay Systems and What Fixed Each One

A perspective post from inside the actual mistakes, not the polished after-the-fact framing. Each item below is something we got wrong on a real project, what happened as a result, and the design change that prevents it on every project since. The pattern is that none of the mistakes were exotic; all of them were the kind of "we'll handle that later" decision that compounded into a problem that was harder to fix later than it would have been to address up front.

We share these because we know other teams have made the same calls. The lessons are not unique to 137Foundry, and the fixes are not proprietary; they are the ordinary design discipline that any team can adopt once they have decided to take replay seriously.

Photo by Liam Briese on Unsplash

Mistake 1: trusting the upstream system's event buffer for recovery

Early on, we built an integration that did not have its own event log. The reasoning was: the upstream system buffers events for seven days, so if we go down for anything less than a week, we can just ask the upstream to replay.

This worked until the upstream system, without warning, reduced their buffer retention from seven days to three days. We found out during an incident that exceeded three days. The replayable events were the ones from the last three days; everything older was lost.

The fix: every integration we build now has its own per-event log on our infrastructure, indexed by upstream event ID, with our own retention policy that we control. The upstream system's buffer is a nice-to-have for cross-checking; it is not the source of truth.

Mistake 2: idempotency keys computed over the entire payload

A second early project used a SHA-256 hash of the full event payload as the idempotency key. The reasoning was: the same event should produce the same hash, and the hash detects duplicates.

This worked until the upstream system added a new optional field to the payload. The hash changed for every event that included the field. Our duplicate-detection check returned false. We reprocessed thousands of events that had already been processed during the partial-availability window of an unrelated outage, producing duplicate downstream records that took two weeks to clean up.

The fix: idempotency keys are now computed over a deliberately-chosen subset of fields, with the algorithm and the field set treated as a contract. New optional fields in the upstream payload are ignored at the hash step. Schema-version prefixes (v1:hash(...), v2:hash(...)) handle the case where the contract has to change. The general pattern is the same one that the Stripe API documentation describes for client-provided idempotency keys, applied to the integration's internal key design.

Mistake 3: building a replay tool without a dry-run mode

A third project shipped a replay tool that did not have a dry-run mode. The reasoning was: the team can read the SQL query that drives the replay and confirm what it will do.

The team could indeed read the query. The query was correct. The replay was correct on the first run. The replay was also correct on the second run, where the operator typoed a single character in the time-window argument and replayed the wrong day's events. The downstream impact was a few hundred duplicate records.

The fix: every replay tool now requires a dry-run mode out of the box, and the dry-run runs by default unless the operator explicitly passes a --live flag. The output of the dry-run is the structured summary the operator reads to confirm intent before running for real. The wrong-day typo would have been caught by the dry-run output showing zero unprocessed events in the window, which is a different number than the operator expected.

Mistake 4: defaulting to the integration's normal throughput on replay

A fourth project's replay tool defaulted to the integration's normal steady-state throughput, on the reasoning that "if the integration can handle it at steady state, it can handle it during recovery."

This was wrong because the downstream system had been handling steady-state traffic during the outage, but the burst of replay traffic on top of that pushed it over its connection limit. The replay finished, the downstream fell over, and we now had a second outage to recover from.

The fix: every replay tool now defaults to a rate limit that is conservatively below the integration's normal steady-state rate, often three to five events per second when the integration normally runs at ten. The operator can opt into higher rates if the situation requires it. The default protects against the case where the operator does not have time to think carefully.

Photo by Diego F. Parra on Pexels

Mistake 5: no dashboard for the event log

A fifth project shipped the per-event log and the replay tool but no dashboard. The reasoning was: operations can SQL the log when they need to, and we will build a dashboard if it becomes a problem.

It became a problem during the first incident. The on-call engineer, at 2 AM, did not know the schema of the log table, did not have a saved query for "events with outcome != success in the last six hours," and ended up writing the query inline and getting it wrong twice before the third version returned the right count.

The fix: every integration ships with a small dashboard that surfaces the event log's recent ingest counts, processing outcomes, and any backlog. The dashboard has a saved "find unprocessed events in this time window" query. The on-call engineer does not have to remember the schema; they click a button. The dashboard takes a day to build the first time and most of that day's effort is reusable across projects.

What these five mistakes have in common

Every one of these was a "we'll handle that later" decision at design time. Every one of them turned into a real production incident that cost more to recover from than the original design work would have cost to do correctly.

The pattern is the same across the five: the design decision looked optional at the time, looked like overkill in the absence of an incident, and looked like the obvious right call in the presence of an incident. The discipline we have ended up codifying is to make all five decisions on day one, regardless of how simple the integration appears. The marginal cost is small. The downside protection is large.

The deeper framing here is that reliability work is bought, not earned. Every reliability feature has to be built explicitly; none of them appear automatically because the team "cares about reliability." The teams we work with that consistently have clean recoveries are the teams that built the recovery tooling before they needed it. The teams we work with that have incident-driven cleanups are the teams that built the recovery tooling after they needed it, which is to say after the first incident has already happened.

For more on this kind of integration reliability work, including the full design walkthrough for the replay tooling itself, the 137Foundry data integration replay guide covers the event log, the idempotency key design, the dry-run mode, and the operational controls. The 137Foundry data integration service page has the broader context for how we approach integration projects, and the rest of the 137Foundry articles cover related reliability patterns like schema drift, error queues, and idempotency for retry-prone webhooks. The underlying technical references are well-grounded: the Wikipedia article on idempotence is the canonical math reference, the Apache Kafka documentation is the standard reference for the durable-log architecture, and PostgreSQL is what most of these event logs ultimately sit on top of.

[137-foundry]

Six Tools We Use Internally for Tracking SaaS Vendor Health and Renewals

Quick post from inside 137Foundry. We run on a stack of about twenty SaaS vendors and we help clients evaluate vendor stacks regularly. The tools below are the ones that earned a permanent spot in how we manage that, listed in order of how much value they deliver per hour we spend with them. None of these are sponsored. None are affiliate links. Some are free, some cost money, some compete with each other. Pick whatever fits.

1. A shared vendor registry in Notion (or Airtable, or a spreadsheet)

This is the foundational one. A single source of truth that lists every SaaS vendor we use, with for each one: contract start date, renewal date, annual cost, primary owner, escalation contact, contract notice period, last evaluation date.

We use Notion because the team already lived in it for other reasons. Airtable works equivalently. A Google Sheet works for a small team. The specific tool matters less than the discipline of keeping it current.

What this catches: renewals coming up that nobody flagged, contracts where the auto-renewal notice period is shorter than we remember, contract owners who have left the company. It is the most boring item on the list and probably the most useful. The Notion templates gallery has reasonable starting templates if you do not want to build from scratch.

2. A status page aggregator (homegrown, ~200 lines of Python)

We wrote this ourselves because the commercial options cost more than the engineering time we saved building it. It polls every vendor status page every two minutes and surfaces degraded status into a Slack channel. The longer guide on building one of these covers the technical details.

What this catches: vendor outages we would otherwise not notice until customers complained, or until an engineer spent an hour debugging the wrong layer. The pattern across the team is consistent: "wait, is X down?" gets answered in five seconds instead of fifteen minutes of debugging.

3. A vendor scorecard template in Google Sheets

The scorecard template walks through eight dimensions of vendor risk: security, data handling, reliability, financial health, integration depth, switching cost, roadmap alignment, support quality. The longer writeup on how 137Foundry approaches procurement covers what each dimension contains and how to weight them. Every vendor evaluation worth more than $25K annual gets one of these filled in.

What this catches: feature-driven decisions that ignore financial-health risk, security-driven decisions that ignore switching cost, all the asymmetric blind spots an evaluation falls into when it is not structured.

4. A renewal calendar with 90-day lead times

Most SaaS contracts auto-renew. Most auto-renewals require 30 to 60 days notice to cancel. The renewal calendar fires alerts 90 days before each renewal date, which is enough lead time to run a structured re-evaluation, negotiate price, or migrate.

We use Notion calendar views with alert rules. Google Calendar works. The discipline matters more than the tooling. A renewal that catches us by surprise gives us no leverage and forces us into a defensive contract negotiation. A renewal we prepared for three months out lets us walk away credibly, which usually leads to better pricing even if we end up renewing.

5. A "vendor health" rotating agenda item in monthly engineering leadership

Once a month, in our engineering leadership meeting, one person walks through the vendor registry highlighting: vendors with upcoming renewals, vendors that had incidents this month, vendors that had pricing or product changes, vendors flagged for re-evaluation. The discussion runs 15 minutes and surfaces issues that would otherwise sit in someone's mental queue.

This is a process rather than a tool, but the discipline is the same as the others. Vendor health goes from "something we should pay attention to" to "something we actually pay attention to" when it has a recurring slot on someone's calendar.

6. Independent review platforms as a sanity check

When evaluating a new vendor, G2, Gartner Peer Insights, and TrustRadius get checked for recent reviews. Filter by company size similar to ours (or to the client's). Read the negative reviews carefully. Look for patterns in what reviewers complain about.

What this catches: the gap between vendor marketing and lived experience. Vendor pricing pages do not mention the support team's average response time; vendor case studies do not mention the migration pain; vendor demo videos do not show the failed integrations. Reviews do.

What did not make the list

A few categories of tool that we have tried and de-emphasized:

Spend management platforms (Vendr, Tropic, etc). These are excellent for companies with serious SaaS spend (typically $1M+ annual across the stack). For a 137Foundry-sized operation, the platforms cost more in license and onboarding than they save in negotiation outcomes. They become valuable around the $2M annual stack threshold.

SaaS management platforms (Zylo, BetterCloud, Torii). Same story: useful at scale, overkill at our size. We get most of the benefit from the homegrown registry plus the renewal calendar.

Vendor due diligence platforms. A few startups in this space try to automate vendor security questionnaire responses. The output is often shallow and the time saved is small. We have not adopted any of them yet.

How this connects to our procurement framework

The six tools together are the operational layer underneath a procurement framework. The framework itself (eight-dimension scorecard, structured re-evaluation cadence, exit plans in every memo) is what 137Foundry uses to make decisions. The tools are what make the framework actually happen instead of being aspirational.

The reverse is also true: without the framework, the tools are just busywork. A renewal calendar that fires alerts nobody acts on is worse than no calendar at all, because it teaches the team to ignore vendor signals. A scorecard nobody fills in is worse than no scorecard. The tools and the framework reinforce each other; either alone is incomplete.

For broader context on how we structure the actual evaluation work, the full vendor scorecard guide covers the framework side in detail.

A few useful external references

NIST cybersecurity framework for the security risk vocabulary

Cloud Security Alliance CCM for cloud-vendor-specific dimensions

G2 Crowd for independent reviews

Gartner Peer Insights for enterprise-focused reviews

What we are still working on

A few things we have not yet figured out:

How to integrate vendor SLA compliance tracking with our own observability so we can argue convincingly when a vendor misses their commitments. The current approach is manual; we want it automated.

How to share scorecards across client engagements without leaking confidential vendor evaluations between clients.

How to factor in newer vendor categories (LLM API providers, vector databases, AI infrastructure) where the evaluation criteria are still evolving and our scorecard dimensions feel slightly off.

If anyone has solved these problems, we are interested. The space is moving fast and the playbook from three years ago does not cover today's vendor categories cleanly.

Setup time, in case you are wondering

The total setup time for the six tools above, starting from a blank slate, was roughly one engineering week distributed over a quarter. The vendor registry was a half day. The status aggregator was two days. The scorecard template was a few hours. The renewal calendar was less than an hour. The monthly meeting slot was a calendar invite. The review-platform checks are not really a tool, just a habit we built.

Spread out the way we did it, none of these felt like a project. They felt like small fixes to specific frustrations. The compounding payoff over the past year has been larger than the upfront work, which is the pattern that makes this kind of investment worth recommending. Almost any team can replicate the basic setup with a few hours per week over a quarter and see meaningful improvement.

[137-foundry]

Notes From Our Last Three SaaS Procurement Decisions

Quick post from inside 137Foundry. We have been on the recommending side of a lot of SaaS procurement decisions over the years, both for our own stack and for client engagements. The last quarter had three decisions worth writing about, because each one taught us something we did not fully appreciate before.

These are the unvarnished notes. Vendor names redacted because we are still mid-relationship with two of them, and the third does not need a public callout. The patterns are what matter.

Decision 1: replacing the data integration tool we had used for four years

The incumbent vendor in this category had been steadily eroding in three ways: pricing was up about 40 percent over the four years we used them, the API was incrementally degrading (deprecations without long notice periods), and the support response times had drifted from 4 hours to 36 hours over the same period.

We evaluated three competitors. The winner was 15 percent more expensive than the incumbent on the per-month line item, but the engineering hours we were spending working around the incumbent's quirks were burning more than the cost differential each month. Total cost of ownership flipped immediately.

What we learned: we should have run the structured evaluation 18 months earlier. By the time we ran it, we had already absorbed about $40K of unbilled engineering time keeping the old integration alive. Earlier evaluation cycles would have caught the deterioration in time to switch without that overhang.

This is a pattern: SaaS vendors degrade gradually after they reach product maturity, and the degradation is invisible from inside an existing relationship until it has been priced in. We are now scheduling a structured re-evaluation of every vendor in our top ten spend categories every 18 months instead of every 36. The longer guide at 137foundry.com walks through the scorecard we now use for these evaluations.

Decision 2: choosing an identity provider for a client engagement

The client was a 150-person SaaS company that had been getting by on a homegrown auth system. They needed to move to a proper identity provider for SOC 2 reasons and because their enterprise customers were asking for SAML SSO.

We ran a comparison between two of the major identity vendors and one smaller, less well-known competitor. The big vendors were strong on integrations, brand recognition, and feature completeness. The smaller competitor was strong on pricing, simplicity, and a more focused product.

The client picked one of the big vendors. The decision came down to one dimension: the smaller competitor had been founded only four years ago and was post-Series-B but pre-profitability. The big vendor had been around for over a decade with a strong financial position. For a five-year identity infrastructure commitment, the financial-health weighting moved the decision decisively.

What we learned: the financial-health dimension of vendor evaluation gets undervalued by engineers (who tend to weight feature completeness and developer experience) and weighted appropriately by procurement (who tend to weight contract terms and pricing). Both perspectives are partial. The structured scorecard approach forces both to be part of the conversation, which is what we wanted but had not always achieved when running evaluations more loosely.

Decision 3: replacing a marketing automation tool

This one was for our own internal use. The incumbent had been acquired six months earlier by a larger company in an adjacent category. The roadmap had quietly shifted from "deep marketing automation" to "marketing module in a larger suite." Two of the features we used were marked as deprecated with an 18-month sunset.

We did the evaluation. Picked a competitor, migrated, finished the move within the deprecation window. Total time to migrate was about 100 engineering hours, plus 40 hours of content rebuild on the marketing side. The migration was painful but the alternative (waiting for the deprecation to force the issue, then migrating under time pressure with no leverage) would have been worse.

What we learned: vendor acquisition is one of the underrated risks in SaaS procurement. The financial-health dimension catches it partially (acquirers often surface in public news months before the deal closes), but the deeper question is "what is the acquirer's incentive to maintain this product?" When a larger company acquires a smaller SaaS, the smaller product is often a stepping stone toward something else, and the parts of it that do not fit the acquirer's strategy get sunset.

Our current heuristic: any vendor that has been acquired within the last 12 months goes onto an annual re-evaluation cycle, regardless of how well the product is otherwise serving us. We are not panicking; we are just making sure we have an exit plan that does not require a 90-day fire drill.

Three patterns from these three decisions

A few things became clearer after writing this up.

Pattern 1: re-evaluate more often than you think you need to. SaaS vendor quality is a moving target. Pricing changes, support degrades, roadmaps shift, acquisitions happen. The version of a vendor you signed with three years ago is rarely the version you have today. An 18-month re-evaluation cadence on top-spend vendors costs a few hours per quarter and surfaces problems while you still have leverage.

Pattern 2: weight financial health more than your engineering team wants to. Engineers are biased toward picking the most exciting product. Procurement is biased toward picking the cheapest one. Financial health (which favors the boring, well-established option) is the dimension that both biases tend to underweight, and it is also the dimension that catches the most expensive procurement mistakes.

Pattern 3: write the exit plan before you sign. Every SaaS vendor relationship ends eventually. The vendor goes out of business, gets acquired, raises prices past your willingness to pay, or simply stops serving your needs. The migration plan you write before signing is far better than the migration plan you write under time pressure when the relationship has gone bad. We now require an "exit plan" section in our vendor evaluation memos. The section is usually three paragraphs and takes 20 minutes to write. It saves orders of magnitude more time later.

What we are doing differently now

Concrete changes we have made in the last six months:

Quarterly review of every vendor in our top-ten spend categories

Structured 8-dimension scorecard for every renewal worth more than $25K annual

Exit plan paragraph required in every initial vendor memo

Status page aggregator that surfaces vendor reliability into Slack

Rotating "vendor health" agenda item in our monthly engineering leadership meeting

None of these are sophisticated. They are mostly forcing functions to ensure we actually do the things we already knew we should be doing.

A few external references we lean on

AICPA SOC 2 framework for the security dimension

NIST cybersecurity framework for general risk-management vocabulary

Cloud Security Alliance CCM for cloud-vendor-specific risk

Gartner Peer Insights for independent vendor reviews

What this is and is not

This is three data points, not a study. Different industries have different risk profiles. Different company sizes mean different procurement constraints. Some of what we do at 137Foundry would be too heavy for a five-person startup and not nearly thorough enough for a Fortune 500.

But the underlying patterns (re-evaluate more often, weight financial health appropriately, write the exit plan before signing) seem to hold across most company sizes we have worked with. If your team is making SaaS procurement decisions without a structured framework, adopting even a lightweight one will probably surface things that surprise you. The exercise is worth doing, regardless of which framework you end up with.

[137-foundry]

When We Recommend Log-Based CDC vs Trigger-Based at 137Foundry

A behind-the-scenes look at the decision we make most often on data integration projects: log-based change data capture or trigger-based. We have done this conversation enough times that the pattern is worth writing down.

The short answer is that log-based wins for most teams once they understand what they are buying, but trigger-based is the right answer often enough that the framing "log-based is always better" is misleading. Worth walking through what we actually think about when a client team asks which one fits.

Photo by Brett Sayles on Pexels

The default position

When we walk into a new client engagement and the question of CDC comes up, our starting position is log-based. Not because it is universally better, but because the failure modes of log-based are operational (which we can mitigate) and the failure modes of trigger-based are structural (which we cannot).

Log-based fails when: - The replication slot fills the WAL and the database runs out of disk. - The CDC tool falls behind for long enough that the source database reclaims the log space. - A schema evolution is not handled correctly by the consumer.

All three of these failure modes are observable, monitorable, and recoverable with the right operational practice. We can set alerts on replication slot lag. We can configure WAL retention to survive realistic outages. We can use a schema registry to handle evolution. The work is real, but it is bounded.

Trigger-based fails when: - The trigger logic has a bug that corrupts the source data. - The trigger overhead degrades source-database write performance under load. - The audit-table cleanup process loses track of which rows have shipped.

These failure modes are harder to recover from because they affect the source database directly. A trigger bug that has been silently producing wrong audit rows for a week is a deeper problem than a log-based pipeline that has been silently lagging. The blast radius is larger.

So we start with log-based and only deviate when something specific about the client situation makes log-based impractical.

When trigger-based is the right answer

Specific client scenarios where we recommend trigger-based over log-based:

The source database is a managed service that does not expose replication-level access. Some managed databases on smaller cloud providers do not give you the ability to consume the log directly. If the database is on a provider that does not offer logical replication, log-based CDC is just not available. Trigger-based works on any database that supports triggers, which is essentially all of them.

The team does not have the operational headroom to run Kafka or a managed CDC service. Log-based CDC is operationally cheap once you have Kafka and Debezium running, and operationally expensive if you have to set them up just for CDC. For a five-person engineering team with no existing Kafka infrastructure, the trigger-based pattern (write to an audit table, read it on a schedule, ship the changes) is often the better answer simply because the team can run it without becoming distributed-systems engineers.

The throughput is low enough that the trigger overhead is invisible. A source database doing tens of writes per second can take the trigger overhead with no measurable performance impact. At thousands of writes per second, the same trigger overhead becomes measurable and starts to matter. We profile the actual write rate before recommending one or the other.

The team needs a queryable change history for non-streaming use cases. A trigger-based audit table is a normal SQL table. Analysts can query it directly. Reports can run against it. The log-based pattern requires the data to flow through Kafka before it is queryable, which is fine for streaming consumers but inconvenient for ad-hoc analysis.

In each of these cases, trigger-based is not a compromise; it is the better answer for the specific situation.

When log-based is the obvious choice

The flip side: situations where we do not even discuss trigger-based.

Sub-second latency requirements. Trigger-based pipelines have an inherent lag between the trigger fire and the audit-table reader catching the new row. Log-based pipelines can deliver sub-second end-to-end latency. For real-time fraud detection, live dashboards, or operational data syncing, log-based is the only credible answer.

High-throughput sources where trigger overhead matters. A database doing thousands of writes per second under steady load will feel the trigger overhead. We measure it with a load test before going live, and if the overhead exceeds the team's tolerance, log-based is the path.

Existing Kafka infrastructure. If the client is already running Kafka for other reasons, adding Debezium for CDC is a small marginal cost. The Kafka cluster pays for itself across multiple use cases, and CDC is just one more producer.

Complex schemas with frequent evolution. Log-based tools handle schema evolution more cleanly than hand-rolled trigger pipelines. For a source database that changes schema every couple of weeks, the schema-evolution cost of trigger-based piles up.

What we do not recommend

Two patterns we actively recommend against:

Building log-based CDC from scratch. Parsing the PostgreSQL WAL or the MySQL binlog directly is possible. The libraries exist. Teams who do this end up rebuilding Debezium badly. The standard tools have solved the operational problems already; rolling your own gives you control at the cost of two engineers full-time for a year. Almost never the right call.

Mixing log-based and trigger-based on the same database without clear boundaries. We have seen teams run log-based CDC for some tables and trigger-based for others on the same source database, with no documentation of which is which. The next engineer who joins has to spend a week figuring out the layout before they can change anything. Pick one pattern per database when possible. If two patterns are required, document the boundary explicitly.

The conversation with the client

When we sit down with a client engineering team and walk through the choice, the conversation usually goes like this:

We start by asking three questions: 1. What is your latency tolerance? (Sub-second, seconds, minutes, hours?) 2. What is your peak write throughput on the source tables you want to capture? 3. Do you have schema-write access and replication access to the source database?

The answers point us at one of the two patterns most of the time. The edge cases (where the answers do not converge on a clear winner) are usually situations where the team should pilot both and pick based on measured behavior.

A pilot is cheaper than a wrong choice. We typically build a small log-based pipeline on one table and a trigger-based pipeline on another table, measure latency and overhead over a few weeks, and let the data settle the question. The cost of the pilot is two engineer-weeks. The cost of choosing wrong and discovering it six months in is twenty engineer-weeks.

The bigger picture

The CDC pattern conversation is part of a larger 137Foundry data integration engagement, where we look at the broader architecture: source databases, downstream consumers, latency requirements, operational headroom, build-vs-buy on the tooling. CDC is one of the questions; the full picture is several.

The longer reference we point clients to is the guide How to Implement Change Data Capture Without Polling Your Database, which lays out all three CDC patterns (log-based, trigger-based, timestamp polling) with a decision rule for picking between them. The internal decision-making we do on client projects is more nuanced than the article (because we have specific client context), but the framework is the same.

For broader background on the underlying ideas, the Wikipedia entry on change data capture is a reasonable starting point.

The 137Foundry view

The CDC pattern is the first technical choice on most data integration projects, and the choice carries through the rest of the build. Get it right and the rest of the architecture follows naturally. Get it wrong and you fight against the pattern for the life of the system.

We tend to recommend log-based when the operational headroom exists, trigger-based when it does not, and timestamp polling only for low-stakes pipelines where the failure modes are acceptable. Most clients land somewhere in the first two categories. Few land in the third.

The right answer is the one that fits your specific situation. The wrong answer is the one that fits someone else's situation but feels safer because they have written about it. We aim to make the choice deliberately, with the client team in the room, on every project.

That is most of what makes data integration work at 137Foundry different from a recipe-driven approach. The recipes are useful background; the decisions still have to be made specifically for each system.

[137-foundry]

Hello from 137Foundry

We're 137Foundry — a small studio that builds web applications, AI integrations, and the kind of internal automation that makes a business actually run.

This Tumblr is the team blog. The main site at 137foundry.com covers our services and case studies in their finished form; this is the looser space — half-shaped ideas, working notes, problems we hit in client work and how we ended up solving them. The kind of thing that doesn't quite fit on the main blog yet but is too interesting not to write down.

A rough idea of what to expect here:

Engineering notes — odd bugs, unexpected platform behaviors, things we learned from a postmortem

AI and automation perspectives — what we're actually shipping in client work, what's overhyped, what's genuinely useful

Hot takes — opinions on web dev tooling, framework drama, the state of the developer ecosystem

Behind-the-scenes — process posts about how we scope projects, write proposals, structure engagements

We post when we have something to share, not on a schedule. If you're a founder or operator thinking about a build, the front door is the main site or our about page — happy to talk.

If you're a fellow practitioner, follow along and argue with us. We learn faster that way.