Tags

In this post, I discuss an example of a pilot and the learnings. In honesty, the story below combines the stories of three pilots to illustrate the learnings.  

Learnings first – for the busy 🙂

No time to read everything. Learnings are summarized below: 

  1. Pitch the product to every division at the customer you encounter because initially you don’t know who can influence the decision on the pilot.  
  2. Ensure project manager at the customer (and at your company) can get things done.
  3. Keep your executive sponsor in the loop even when the activity is outside their division. 
  4. Have great documentation and self-serve tools for customer’s engineering teams. Treat these just as important as the core product. Stripe does this excellently.
  5. Be humble. The folks at the customer team just did you a huge favor by launching. Say thanks. Send them schwag. Send the engineers your Eng blog.
  6. Update the pilot success criteria based on learnings gathered over the course of the pilot engagement.
  7. Set up a cadence of updates on pilot performance. Build templates for these updates. 
  8. Get the customer to validate results so they believe it. Speak Monte Carlo permutation to the statistician. Speak business case to the executives. Speak design possibilities to UX. Also speak the vision and the opportunity the startup offers.
  9. Have your customer pitch his/her executives rather than you yourself. Take every effort to prep them for the meeting. Make them look great. 
  10. Ask customer to be a reference and a case study, if they are not already committed to in the agreement. 

Now the full story for the more patient 🙂 The lessons above are also included as commentary along the story. Enjoy. 

Customer Situation

The customer was an e-commerce retailer. The primary decision-maker was a Marketing executive – let’s call her Elsa. Elsa’s primary goal was to increase revenue at an attractive cost, so that was our success criteria at the outset. The pilot contract was signed and the first meeting had been scheduled. 

A pilot’s timeline goes through stages from pre-sale to requirements discovery to integration to pilot measurement and close. The story below is written in this same timeline. 

Discover business requirements and finalize scope

We discovered new stakeholders in several divisions such as E-commerce, UX, Engineering, IT, Network Security, Privacy, and Procurement. Some were also from our customer’s parent company. Elsa had assigned a project manager – let’s call him Hans – but he didn’t call the shots with the other divisions. So we had to speak directly with the divisions. We started off being  task-focused, for example we gave IP addresses to and asked for certain QPS from Network Security, but this request was in the queue and Hans could expedite some but not much.  

Learnings at this stage: 

  • Pitch the product to every division at the customer  you encounter because initially you don’t know who can influence the decision on the pilot.   
  • Ensure project manager at the customer (and at your company) can get things done.

Design the solution

The customer’s UX had a concern about a specific aspect of the look and feel of a feature. However addressing it would diminish product performance. We needed to push back. Pitching the product was helpful to the UX team. But it did not change their mind because the UX team was following their mandated internal standards. Finally we spoke to Elsa and she spoke internally and made the call.

Learnings at this stage:  

  • Keep your executive sponsor in the loop even when the activity is outside their division.

Integrate product

Customer’s Engineering team was quite large and it wasn’t clear early on which engineers would work on integrating the product. 

Learnings at this stage:

  • Have great documentation and self-serve tools for customer’s engineering teams. Treat these just as important as the core product. Stripe does this excellently.

The pilot

Launched! A few high fives internally and then we sent a huge thanks to Elsa, Hans, and the other customer teams. Then back to work. The first step was to update and re-confirm the success criteria for the pilot. Then we began to  share regular updates with the customer. A member of their Analytics team validated performance on their own also. For the final update with the executives, Elsa wanted to give the presentation on the pilot – a first for us and we agreed. 

Learnings at this stage:

  • Be humble. The folks at the customer team just did you a huge favor by launching. Say thanks. Send them schwag. Send the engineers your Eng blog.
  • Update the pilot success criteria based on learnings gathered over the course of the pilot engagement.
  • Set up a cadence of updates on pilot performance. Build templates for these updates. 
  • Get the analytics person at the customer to validate findings. Speak Monte Carlo permutation to the statistician. Speak business case to the executives. Speak design possibilities to UX. Also speak the vision and the opportunity the startup brings to the customer.
  • Have your customer pitch his/her executives rather than you yourself. Take every effort to prep them for the meeting. Make them look great. 
  • Ask customer to be a reference and a case study, if they are not already committed to in the agreement.

Here’s the first post on winning pilots. What’s been your experience with pilots? Or have questions? Let’s talk.