Blane Canada

Related question

Can I trust a vendor demonstration?

Short answer

Trust what it proves, not what it implies. Demonstrations look flawless because they run on demonstration data that was packaged to make them look flawless. Your data is different. Ask to see the tool run on a sample of your actual, messy information.

Demonstrations excel at looking good, and that is by design rather than by deception. A vendor prepares clean, well-structured, representative data because that is what shows the capability clearly.

The problem is that the gap between that data and yours is precisely the work you will be paying for, and it is invisible in the meeting.

When we build for clients, we use their actual data from day one. The results are still impressive. They are also honest, because you get to see what it will take to make your information usable and how the system behaves when the input is imperfect.

There is a related pattern worth watching for. I test-drove a prospecting tool recently that had real potential and generated leads quickly with a lot of useful information. The marketing claimed a task took two minutes. In practice it took closer to twenty, largely because there were no parameters for refining a general search. The tool was not misrepresented in what it could do. The time claim simply did not survive contact with a real workflow.

Ask two things of any demonstration. Run it on my data. And walk me through what happens when the input is wrong.

Go deeper