How to Evaluate a Technology Vendor Before You Depend on One
Most software selection processes are built around a feature comparison, usually a spreadsheet with rows for capabilities and columns for candidates. It is a comfortable format and a poor predictor of outcomes, because the features are the part most likely to converge between serious competitors and the part least likely to determine whether the relationship works in year three. The questions that actually matter are harder to score and considerably more useful.
Does the vendor's business model align with yours
Start with how the company makes money and from whom. A vendor whose revenue comes from thousands of small customers will build for the average case and will not take your bespoke requirement seriously, whatever the sales conversation suggests. A vendor with a handful of enormous accounts will build for them, and your roadmap requests will queue behind theirs indefinitely. Neither arrangement is wrong; both determine what you can expect. Procurement guidance that leads with this question, including the buyer's material at the procurement coverage on this site, tends to produce fewer disappointed renewals than guidance organised around capability matrices.
Related: check whether the product you are buying is central to the vendor's business or peripheral to it. Peripheral products get maintained until they are not, and the notice period is usually a blog post.
Viability, honestly assessed
- How long has the company operated, and through what funding stages
- Is the product line profitable, or subsidised by something else
- What proportion of revenue comes from the single largest customer
- Has there been recent leadership or ownership turnover
- What happened to their previous acquisitions, if any
You will not get complete answers, and the quality of the answers you do get is itself informative. A vendor that responds precisely to uncomfortable questions is signalling something about how they will behave during an incident. One that deflects is signalling that too.
The support conversation, conducted properly
Every vendor claims excellent support and most contracts define it in terms that sound reassuring and commit to very little. Response time is not resolution time, and a four-hour response guarantee is satisfied by an automated acknowledgement. Ask instead: who answers a severity-one ticket at three in the morning, in which time zone, and what is their escalation path? How many support engineers exist relative to the customer base? What is the published process when a defect is confirmed but not scheduled?
Then verify independently. Ask for reference customers, and specifically ask for one who has had a serious incident. A vendor confident in its support will produce that reference; a vendor that offers only enthusiastic new customers is answering by omission.
Total cost is never the licence fee
Build the full picture before comparing prices. Implementation and configuration effort, usually measured in your own staff's months. Training for the people who will use it daily. Integration work with everything it must connect to. Ongoing administration. The price of the tier you will actually be on in two years, after growth, not the one you start on. And the cost of the modules that turn out to be necessary rather than optional — a pattern common enough that it should be assumed rather than discovered.
Ask about leaving before you arrive
The exit question is the one most consistently skipped and most consistently regretted. Can you export your data, in a documented format, including history and metadata rather than a flattened current-state extract? Is there an API that would let you leave, and is it complete enough to be useful? Who owns the configuration you build inside the product? What notice applies, and what happens to your data at termination?
Ask these during the sales process, when you have leverage. The answers are informative, the responses are contractual, and the exercise establishes at the outset that you regard the relationship as one you might end. Vendors behave differently toward customers who have demonstrably thought about this.
Pilot against reality
Vendor demonstrations run on prepared data at a scale that flatters the product. A meaningful evaluation runs your data, at something like your volume, with your integrations, performed by the people who will use it rather than by a specialist. Define in advance what constitutes success — specific tasks completed within specific times — and hold the trial to those criteria rather than to overall impression. Impressions are shaped by the quality of the sales engineer, which is not a feature of the product you are buying.
Read the contract for the parts nobody reads
Automatic renewal terms and the notice window required to prevent them. Price increase caps, or their absence. What the service credits actually pay out, which is usually a small fraction of what an outage costs you. Data residency and processing terms if you have regulatory obligations. Sub-processor lists, since your vendor's dependencies become yours. And audit rights, which matter more than they appear to until the day you need to demonstrate something to a regulator.
Security review is not a checkbox
Certifications tell you a vendor has a process, not that the process is good, and a compliance report describes a point in time rather than current practice. Ask what happens when a vulnerability is reported to them: is there a published disclosure policy, what were the last three advisories, and how quickly were customers notified? Ask how authentication integrates with your identity provider, whether administrative actions are logged in a form you can export, and who at the vendor can access your data in the course of support. The answers separate organisations that treat security as an engineering concern from those that treat it as a sales obstacle, and the difference becomes extremely visible during an incident.
Write down why you chose
Finally, record the decision: the alternatives considered, the criteria weighted, the assumptions made, and the conditions under which you would revisit. Two years later, when someone asks why the organisation is on this platform, that document is the difference between an informed reassessment and a rewrite driven by whoever is newest and most frustrated. It also disciplines the original decision, because assumptions written down are noticeably harder to hold than assumptions merely felt.
