Organizations often choose technologies for short-sighted reasons. Here’s a better way with 3 questions to ask and 10 criteria to evaluate.
When I look back at our last decade and more than 500 social sector technology projects (and the systems we’ve replaced), it strikes me that organizations often choose technologies for silly and short-sighted reasons. These range from “my cousin works at Oracle” to “everyone here uses Excel, so of course we’re going with Sharepoint / Dynamics / PowerBI” to “the maps look really cool” to “their hotline is open 24/7” to “we can’t afford any license fees” to “we only use open-source technology / pro-bono vendors”.
We’ve honed in on a common-denominator model of three questions organizations should ask and ten criteria they should evaluate when choosing a technology and have shared it with dozens of clients and partners. The model seems to fit big and small organizations, cloud and mobile technology, and a range of system types (CRM, ERP, MEL, portfolio management). The feedback has been so positive that we’ve decided to publish the model here so others can use it (and help us improve it).
Question 1: To what extent will this technology meet our needs today?
While often overweighted in nonprofits’ technology selection, this question is critical because it shapes how well the tool or platform will deliver business value in the short-term and, by extension, end-users’ early impressions of the technology. To answer this question, break it down into three criteria:
1. Features: Does the technology provide out-of-the-box features that meet your requirements? If not, can the gaps be filled through configuration?
List out all the requirements you can currently imagine addressing with the new technology, grouping ones that are related and categorizing into buckets like Automation, Analytics, Mobile, etc. Then identify each requirement as ‘must-have’, ‘should-have’, or ‘could-have’, making sure it’s described specifically enough (e.g. “Automatically send email reminders when reports are overdue” or “Display data in a point map”). Then get vendors to honestly score their technologies as follows: 3 points if the feature is provided out-of-the-box, 2 points if the feature can be configured without code, 1 point if the feature can be delivered through custom development, 0 points if the requirement cannot be met. You don’t have to have a hundred-million-dollar annual budget or be running a 6-month competitive RFP to do this exercise.
2. User-friendliness: How intuitive and easy-to-understand is the technology’s user-interface and user-experience?
You might have strong opinions yourself, but end-users are best positioned to evaluate this criterion. It’s important to look at user-friendliness through each lens of the technology. Often we see user-friendliness evaluated only from the perspective of end-users doing data collection or capture – that’s a crucial lens, but the technology’s friendliness for data management, data analysis, data visualization may be equally important.
3. Security: Can you trust that your data is safe on the technology?
This criterion is not simply about ticking a box that the technology provider has a white paper about GDPR or ISO 27001 – when it comes to data protection, GDPR compliance has as much to do with your practices as with the technology you use. Rather, think about security through two lenses: internal and external.
For internal security, evaluate how well you’ll be able to control users’ access and permissions with the technology, ensuring the right staff are able to see and do the right things. Look for technologies that allow you control access and permissions at object/table-level, at feature-level, and at field-level. You’ll also want an audit trail that helps you track who made which changes and when (both for data and metadata). For external security, you want to ensure your system will be safe and that only authenticated users will have access. You might want features like: single-sign-on, two-factor-authentication, and the ability to insist on password requirements. And assuming you’re storing data on the cloud, you’ll certainly want a technology provider with a strong reputation for how it manages its servers both physically and digitally.
Question 2: To what extent will the technology meet our future needs?
This question is almost always more important than the first question, yet often receives less attention in the selection process. If you ask anyone at Vera Solutions why we build on the Salesforce Platform, nearly everyone’s answers will revolve around the platform’s sustainability and never-ending improvement. Again, break this down into four criteria.
4. Flexibility: How easily can the solution evolve as your organization and your requirements evolve? Try to picture 5 years from now – is the technology propelling new ideas and ways of working or is it struggling to keep up as the organization changes and matures?
To assess a technology’s flexibility, you’ll need to understand, for instance:
- How configurable are its automation features? Do business rules need to be hard-coded or can they be managed through modular, adaptable automation tools.
- How configurable are its analytics features? Do reports and dashboards require customization or can they be adapted in a drag-and-drop way by end-users?
- How easy or hard is it to adapt or extend the underlying data model? Over time, you will certainly need to capture new data points, modify picklist values, inactivate certain fields, introduce entire new data tables – do you need to rely on developers to make these sorts of commonplace changes or will you be able to make them in-house?
- Can your organization extend the technology to handle major new requirements and new entire use cases that may come up in the next 5 years?
If you think about a solution you are still thrilled with five years from now and work backwards, one of the most important characteristics to optimize for is interoperability, or how easily the tool can integrate with other tools. On an interoperability scale from 1-10, a ‘1’ would be a tool on a custom, proprietary stack with no documented API and few examples of successful integration to date, while a ‘10’ would be a technology with an extensively documented set of APIs, a wide range of plug-and-play integrations, and myriad published examples of successful integrations including with tools your organization uses. Investing in a middleware can help speed up integration efforts and reduce the cost of maintaining integrations over time.
The best SaaS companies invest heavily in R&D, ensuring their products stay ahead of the game when it comes to features, user-friendliness, security, flexibility, and interoperability. In answering the question of meeting future needs, don’t just consider what a tool or technology offers today, extrapolate from its innovation track-record to anticipate what it will offer in the future. To evaluate innovation, you’ll want to know how many releases per year to expect, what the current high-level roadmap looks like, how the roadmap is prioritized, and what kind of influence you’ll have (as a customer) on the roadmap.
7. Ecosystem: How strong and connected is the community of users and partners around the technology? How widely available is information that will help you troubleshoot or improve your implementation of the technology?
Perhaps the most underrated criterion to evaluate is the ecosystem of customers, partners, and knowledge surrounding a technology. On one side of the spectrum, you may have a brand new tool with shiny features but only a few customers, a few developers who can adapt it, and a few online resources. On the other side of the spectrum, you have platform solutions, like Salesforce, with a huge and well-connected online community providing help and training, hundreds of partners globally, 150,000+ customers, and thousands of hours of free, gamified online learning modules on Trailhead.
Question 3: To what extent does the technology work within your budget?
Depending on how spending works in your organization, it may be tempting to look at this question purely from the perspective of “how do we minimize ongoing license fees?” or “how do we get a solution up and running on a budget of X?” Making a sound, strategic decision hinges on understanding the technology’s total cost of ownership (TCO). TCO breaks down into three components: setup costs, license costs, and maintenance costs. (Note: If you want a much more detailed guide on working out TCO, check out Brightwork’s methodology.) Some of these will be tangible, direct costs; some will be indirect like staff time, staff motivation, or staff turnover. For marketing and fundraising systems, you will want to look not only at the cost, but also at the expected return-on-investment; for MEL and program technology, the return-on-investment tends to be far less tangible and harder to calculate.
8. Setup Costs: How much will it cost us–directly and indirectly–to design, configure, and rollout this tool?
Firstly, work out what upfront investment will be needed to get the tool up and running. A familiar tool that requires little or no configuration (e.g. Google Docs or Box.com) will require much less investment than a tool that requires months of complex setup (e.g. a CRM or M&E system). Setup costs not only include consulting services required to tailor and train, but also internal staff time required to help design, test, and adopt the tool. Setup costs will vary from provider to provider and will depend on whether you’re using a fixed-price or time-and-materials model. Unless your organization has developed firm, mature requirements or you need a very simple, straight-forward solution, a capped and carefully managed time-and-materials contract with a partner you trust is usually the most reliable model for ensuring a win-win project.
While many great tools offer a ‘freemium’ model, most technology that is worth using is licensed. While many in the nonprofit sector have been taught to believe that open-source is always better, the costs of relying on open-source tools often outweigh the benefits. The advantages of licensing software are similar to the advantages of renting an apartment instead of crashing on your friends’ couch. When it comes to Salesforce, most nonprofits are eligible for 10 free licenses, but remember that Salesforce is only ‘free’ like a free puppy is free.
10. Maintenance Costs: What will it take and what will it cost to support this technology?
The final and perhaps most important component of answering the budget question revolves around the cost of maintaining and supporting the technology once it has been launched. “How is maintenance the most important cost component?” you ask.
Jeff Hanby answers that question well, breaking out examples of the corrective, adaptive, preventive, and perfective maintenance needs that come up and citing five articles that suggest maintenance costs tend to make up 40-80% of TCO.
The more customized your solution is, the higher your maintenance costs will be. Direct costs here may include:
- Staff time from System Administrators (anywhere from 20% to 300% full-time-equivalent, depending on the system’s size and complexity) spent managing and adapting the system
- Support services from the technology provider or implementation partner to troubleshoot, debug, or make minor adaptations (often provided on a retainer basis or as part of a license fee)
- Additional services required to adapt the technology to meet your evolving requirements (mitigated by choosing a technology with excellent flexibility and a strong ecosystem)
- Buying and maintaining servers or any other hardware required to use the technology
- Time or services needed when retiring the software (exporting data, keeping the system running while a replacement is getting rolled out)
Indirect costs, meanwhile, may include:
- The cost of staff frustration, burnout, or attrition resulting from using the unfriendly, inflexible, non-interoperable technology you chose (though it may not show up on the budget, there is a very real cost to your M&E Officers ripping their hair out)
- The inefficiencies and time wasted by relying on manual data manipulation and having easy answers to common questions at your fingertips
- The cost of uninformed decisions made because your decision-makers don’t have the data or information they need when they need it
