January 30, 2006

Show Me the Metrics!

Posted in customer service, Economic Model, Execution, management, Metrics, Sales at 11:57 pm by scottmaxwell

About 10 days ago I interviewed a VP Sales candidate for one of my portfolio companies. The more I think about our conversation, the more I like him. I walked out of our conversation thinking that he would be perfect for any B2B software or SAAS company. Why?

This particular candidate pulled out his personal operating report and explained to me how he knows what is going on every day (even when he is on the road with customers) and what he does when particular metrics drop below a certain level. He showed me the different levels of his report so that he could see his deals moving through the sales pipeline and how each salesperson and sales group was doing vs. their benchmarks. He also explained how predictable his system is in forecasting sales as well as determining when he needed to add resources at various stages of the sales process. Finally, he was able to explain how his system allows him to accurately predict the results of adding new resources and, just as importantly, how he is able to relatively easily recruit and train people to follow his system. Separately, he has relevant experience and success managing several models of distribution (including telesales, field sales, and channel sales), each of which has its own unique best practices. Also, we were able to get a reliable reference that echoed what the candidate had said.

He did not quite “have me at hello,” but he did have me about 10 minutes into our discussion. I knew going in that he had relevant experience and was known for making his numbers, but it was his approach of managing to metrics that got me. Why?

When you can accurately predict your results in each operating unit, it means less risk and a greater opportunity to scale your company without blowing your capital (missed quarters get more and more expensive as you grow!). Being able to make accurate predictions also means that:

  • You have an operating model (not just a collection of people), which allows you to scale better,
  • You understand the key drivers of output in your operating model,
  • You are consistently managing the unit to your operating model,
  • You have a set of early warning signs (your key drivers) that you can focus more attention on when they get below certain thresholds (i.e., it helps you to know where to spend your time),
  • You have a set of measures that you can benchmark against other companies to understand where you have opportunities to move to best practices, and
  • You know when you need to add staff or other resources well before you get caught short.

Finally, the understanding of the above gives you a solid platform for experimenting with new approaches and accurately evaluating the effectiveness of the new approaches (thereby allowing you to kill the approaches that don’t work and expanding the approaches that do work).

Over time, the nature of emerging growth companies is that they move from simpler approaches to more sophisticated approaches (more specialists, more channels of distribution, more products, more marketing channels, more approaches to customer service) and you want to make sure that you continue to evolve in the right direction (note: this is not an argument to get more sophisticated as an end to itself, just that getting more sophisticated leads to better operating results as you growl…you clearly need to keep your operation as simple as possible).

Metric driven management can and should be applied in every functional unit in an expansion stage company, from product development activities (e.g., project management, bug fix reports, usability testing) to marketing (lead generation ROI, website path analysis, shopping cart abandonment, number of daily quality leads) to sales (e.g., movement through the sales funnel, salesperson activity analysis) to customer service (response time, close rates, close times, etc.) to overall customer satisfaction measured both qualitatively (surveys, interviews, etc.) and quantitatively (usage reports, retention rates, etc.).

The key to getting the right metrics program in place is to eventually understand the minimum number of measure that give you an accurate understanding of the state of your company.

Some Caveats:

  • Many (most?) very early stage companies can get by without metrics-based management, as there are very few people in the organization, the processes you have are quite simple, and you can manage staff a lot easier. But as soon as you start getting any measurable number of users/customers, metrics-based management starts becoming useful, and as you grow more metrics become difficult to live without.
  • There is no sense building systematic operating models and a set of metrics if you are not going to manage to them. I have met many intuitive managers who don’t get (or don’t want to get) this approach. If you don’t believe in the approach, shoot me a note or comment to this post. If you don’t completely get the approach, hire someone to work for you who does (I have done this multiple times at my portfolio companies).
  • Once you lock into a set of metrics (it will take some time to determine the best most simple metrics), you should try to use the same metrics over time. I am amazed when I go into certain board meetings and see a different set of metrics each quarter…sometimes managers feel the need to present the metrics that show off the accomplishments of the company…I would rather see the metrics that show the improvement opportunities for the company…this is where the real upside is!


If you want to tell me about your results from or projections for your operations, my preference is that you show me the metrics!

Are You Building an Army of Evangelists?

Posted in culture, customer service, marketing at 8:33 am by scottmaxwell

While large companies have had a growing trend toward hiring “technology evangelists,” it is becoming clearer to me that institutionalizaing the approach (i.e., setting goals, formally staffing the roles, responsibilities, processes, metrics) has increasing benefit in emerging growth technology companies.

What is “Evangelism”?

James Pethokoukis has a good U.S. News overview article, Spreading the Word, on the approach used by several companies (you have to sit through an annoyingly long 10 second ad to see it, but the article is worth it).

In my view, the key issue is all about how the emerging growth company organizes to leverage available communication channels and external influencers to both “get the word out” and, possibly more importantly, “get the word in!

Why Now?

This probably does not take a lot of explanation, but the increased number of impressions formed by online activity (forums, blogging, domain-specific directories, etc.) makes the benefit associated with evangelism much higher for all companies, and the positive ROI now extends down to extremely small companies.

Some Viewpoints…

Guy Kawasaki, the first technical evangelist when he worked for Apple, has a recent post The Art of Evangelism. Declan Elliott has some good thoughts and links on the topic. Guy’s approach is a little too “religious” for my taste, but it works for him as well as the causes he evangelizes (you can, of course, tune the message to your customers’ taste).

Robert Scoble is probably the best known current Evangelist, working for Microsoft. He has a follow-up to Guy’s note with his point of view. The comments are also a great read (note the different viewpoints on the use of the “evangelism title vs. other possible titles for the activity). Another must-read is Robert Scoble and Shel Israel’s new book, Naked Conversations (reviewed here), which gives a great overview on the use of blogging for creating conversations with the people that make up your market (not sure about the title however…the image of Robert Scoble naked is not appealing to me).
Church of the Customer Blog has some of their thoughts on customer evangelism vs. corporate evangelism (Ben and Jackie also offer up their books and other material on making your customers evangelists. I have read them and found them to be full of great practical ideas for building customer evangelists…their blog is well done and on my reading list too…

How to get started…

Ben and Jackie have some practical ideas on getting started in their post How to become a Customer Evangelism Evangelist The key here is just to get started in some capacity…even by just opening your eyes to the opportunity. See this view from a emerging growth company from 2% Creativity Blog.

January 27, 2006

Are you Nickel and Diming Your Customers?

Posted in customer service, management at 9:47 am by scottmaxwell

I have been traveling internationally this week (one of the reasons for my sparse postings) and I had an interaction as customer that made me examine how not to train your people.

Briefly, I was at the Airport in Munich Germany waiting for my flight to Moscow. I purchased a cigar at a shop using a credit card (I did not have any Euros, only U.S. cash). Then, I asked for some matches. The woman behind the counter told me that the matches were $0.10 Euro (a dime). I still did not have Euros and the credit card had already gone through for the cigar. I offered either U.S. cash or my credit card again.

At this point, the woman behind the counter could have offered to give me the matches, taken my U.S. cash, or even taken my credit card. Instead, she told me she couldn’t take the credit card for such a small purchase, wouldn’t take U.S. cash, and didn’t offer to give me the matches (I didn’t ask either, as I wanted to see what she would do). I walked out of the store with a cigar and nothing to light it with.

Needless to say, the lack of matches left a bad feeling for the shop, the airport, and even the country that was several orders of magnitude greater than the price of the matches…

What I took out of the experience:

1. If you train your customer service staff on policies and procedures, they are likely to follow them (e.g., matches cost money, don’t accept U.S. dollars, no credit card purchases below a certain threshold). Perhaps interjecting some principles (e.g., make sure the customer is treated well, don’t nickel and dime!) and allow your staff to break the rules when the result from the rules seems to go against the principles. This issue is particularly bad at large companies, but I continue to see opportunities for improvement at many small companies, including this small shop in the Munich airport…

2. If you don’t train your people on thinking about the needs of the customer and how your product might work for them (or not work for them) before the purchase is complete, you might end up with mismatched expectations.

postscript: I am writing this post from 30,000 using Lufthansa’s in flight internet connection. The airline and the country shot up a few orders of magnitude when I turned on my laptop in flight!

Development Methodology

Posted in Product Development at 9:41 am by scottmaxwell

Ryan Martens, CTO of Rally Software Development sent me this a link to Alisair Cockburn’s article The Methodology Space. If you are into development methodology frameworks, this paper has a pretty well thought out and interesting set of thoughts on development methodology and how it needs to change to suit your situation. (warning: not light reading)…

You really need to focus!

Posted in CEO, management, Product Development at 9:14 am by scottmaxwell

I posted several times on the issue of focus, including the opportunity to develop a scope advantage against the large company, the issue of time horizon of CEO focus vs. company size, the the issue of CEO time horizon focus by day of the year.

I am just completing a week of meetings with emerging growth technology companies in Germany and Russia, and the meetings reminded me again that everyone talks about focus, but very few companies actually do focus (this is true of emerging growth companies in all countries, not just the companies that triggered this post).

There are two major opportunities for most (all?) emerging growth companies:

  1. Narrow your focus– Reduce the number of products and the number of new features that you are trying to add to the products. Use the extra time to make the product and features that you are developing that much better (and simpler).
  2. Improve the target of your focus– Listen to your customers to help you narrow down to the right product and the right features. (some ideas on how to do this are in my post on gaining an information advantage).

Ok. I know…obvious points. But if the points are so obvious, why is it that so many companies feel that they understand their customers so well without spending much time with the customers? Why is it that they are thinking up grand new products when their current products have a long way to go before they are fully intuitive and extremely easy to use?

I think the issue is that most companies are not critical enough of themselves…they go through their day thinking that they know their customer (using logic and a lot of assumptions) and they are focusing when EVERY company has significant opportunity to improve on both! (one senior manager on this trip went so far as to explain to me that talking to the customers will mislead the company into believing the feedback, which would only be relevant for that particular customer. While he is right to point out that you need to be careful about your approach and conclusions that you reach, this is a very bad excuse for not taking the time to understand your customers’ point of view!!)

An approach that will help most (all?) companies is to appoint a person responsible for both formalizing and capturing the customers’ feedback AND be responsible for minimizing scope creep (this needs to be a highly disciplined person that feels comfortable keeping everyone focused and on target). The role could be called (or be part of) product marketing or product management. If you do not have this role in your organization, consider creating it and assigning the right type of person to it.

You have a huge opportunity to make better products while growing faster with fewer managment headaches.  At a minimum, walk around your company for the rest of the day saying to everyone that you meet “You really need to focus!”  I am 100% certain that everyone will agree…

January 16, 2006

Search Tech Opportunities- Summary

Posted in innovate this!, Innovation at 8:28 pm by scottmaxwell

This is a summary of my series on Opening up the Search Tech Chain. The major point is that opening up the search technology model will have some great effects on all of the possible innovations around the model. The post the opening of the search tech chain discusses my argument for the opening up of the tech value chain and links to some other bloggers thoughts and resources on OpenSearch and the opening of the Alexa A9 search engine platform for others to innovate on.

Just so that I am clear on the abundance of innovation opportunities related to search, I have several posts that describe some opportunities for innovators. The ideas overall can be applied to text, images, audio, or video, each of which has its own issues and opportunities.
The posts:

  1. Every site needs to expose its APIs. Open APIs would be a huge opportunity to improve search, even with today’s search technologies.
  2. Lots of innovation potential in micropayments. While the ecosystem could get started without a micropayments infrastructure, these issues will need to get worked out.
  3. Mashups will be a lot more interesting with the new APIs. In my view, the mashups for the end users are the most exciting innovations (but probably not the most difficult). I lay out my thoughts on the potential for mashups in this post.
  4. Help me find what I am looking for. The matching of search intent to search results works okay today, but there are lots of improvement opportunities.
  5. Some thoughts on improving tagging. The social tagging sites have changed my use of the internet in tremendous ways. I point out some of the improvements to the tagging that I would like to see in this post.
  6. Put more features in feature vectors. We need to move beyond exact word search into many new approaches to finding what we are looking for. The posts start getting a little more technical here with the ideas on creating and exposing larger feature vectors.
  7. Machine Learning as a Web Service. Machine learning is one of developments on the internet that will change things dramatically for the users (and will offer some great economic results to the winners). This post outlines some thoughts on how it might be achieved as a web service.
  8. Machine Extraction. Finally, turning unstructured unlinked data into structured and linked data is unbelieveably difficult but very powerful stuff. This post gives an example and outlines some other rough thoughts on the topic.

I am sure that I have missed lots of opportunities for innovation in search, but this list is still very long and hopefully demonstrates the point that there is a lot to do! If I have any other ideas worth posting, I will post them and link to them here. Please also comment if there are other ideas or resources that are valuable to this list.

Search Tech Opportunities- Improvement #8: Machine Extraction

Posted in innovate this!, Innovation at 7:49 pm by scottmaxwell

Following up my post on the opening of the search tech chain, the eighth opportunity that I see is with machine extraction and, even better, “linking” material from machine extraction. While this is really a subset of machine learning, the concept is different enough to discuss separately. The basic idea is how can you extract information from the web and put it into a more structured and, more importantly, accurate form. Then, how can you infer the relationships between the data elements that you have extracted.

A good example of a company related to this field is zoom info. Their tag line is “the search engine for discovering people, companies, and relationships.” I typed my name in to see what they had on me, and the list of information that they were able to extract and put together was very complete (I saw one error, as they matched me with an Insight Ventures that has a confusingly similar name to my firm. Hard to take that away from them, however). Pretty impressive stuff!

It seems to me that machine extraction and building relationships is an important part of finding non-explicit links between entities (people, places, and things) as well as compiling information on those entities. The web has a lot of resources (URIs) that help describe the entities, but Machine Extraction along with other Machine Learning techniques may be what is necessary to help push forward with Tim Berners-Lee’s vision for the Semantic Web (a very powerful concept).

I would put this in the category of VERY advanced search with an HUGE amount of innovation potential!

Search Tech Opportunities- Improvement #7: Machine Learning as a Web Service

Posted in innovate this!, Innovation at 7:23 pm by scottmaxwell

Following up my post on the opening of the search tech chain, the seventh opportunity that I see is with machine learning of various kinds. If you are not familiar with machine learning, take a look at Tom Mitchell’s book on the topic (funny, when I punched “machine learning” into Google, the first entry was an advertisement from Google asking “Want to work at Google?”)

This topic gets deep and broad very fast. I have put a lot of time into it over the years (starting with some credit card behavioral modeling in the early ’90s), but even with all the research I have done, I know only enough to be dangerous. Machine learning is relatively technical and relatively difficult to get exactly right (lots of math, CS, and art here).

I do, however, have a few non-technical thoughts on the topic:

What I would use it for…

Machine learning can be used for an amazing number of things (too many to describe here and many, many that I have not even considered). With respect to search (and assuming that all of my improvement ideas so far have had some level of development), innovators could create the following (and much more):

  • Propose tags for me on the social tagging (as I requested in an earlier post on tagging).
  • Given a set of resources (a.k.a., webpages, URIs), find resources that are similar (the find similar buttons on the search engines are really quite bad at this point. The approach that I would test would be asking the user what “dimensions” of similarity the user is looking for and then find everything similar. Note that the approach would use the feature vectors and models from the machine learning algorithms.
  • Automatically update my Ajax desktop pico-domain. There is refresh work that would need to be done in addition to the machine learning, but I should be able to develop algorithms that help me to quickly find and update resources in my domain-specific site.

These are just a few of the list of examples that solid machine learning can do. Clearly, it is not perfect, so I will still need to have some manual activity to sort through some bad results, particularly at the beginning. But, the machine learning will save me a lot of time.

An innovative service…

The problem is that building good machine learning models is a time intensive task that starts with creating a model building environment. While many of the large companies appear to be doing just this, most smaller companies are standing flat footed in this area, due to lack of understanding, resources, and skills.

So, how about an innovative On-demand (SAAS) service as an offering in this area from a company that has the skills (generally now operating as professional services groups)? Most internet services could use machine learning of one kind or another, but they do not have the resources/skills to set up the model development environment. The service could be helpful with code that helps create feature vectors (and other machine learning inputs), recommend modeling approaches for the particular class of problems, walk the user through the model building process, and back test the models. Finally, it could deliver the code for the models or, possibly, even implement the models on its own systems as an ongoing service.

Since the inputs are all available via the Internet and there is a lot of work in the set-up of the model building environment, this innovation lends itself pretty nicely to be set up as an internet based service.

Given that the process still has a fair amount of art in it, the service could also offer up expert model building advisors to its customers.

If some innovators do not move in this direction for the community at large to share, this will become a major area of strategic advantage for the larger companies over the next few years. Perhaps Alexa (or another innovator) will move in this direction for the benefit of many?
Huge amount of innovation potential here!

Search Tech Opportunities- Improvement #6: Feature Vectors

Posted in innovate this!, Innovation at 5:27 pm by scottmaxwell

Following up my post on the opening of the search tech chain, the next few opportunities start getting slightly more technical. The sixth opportunity that I see is with more useful features in the search feature vectors and the mathematical combination of entries in those vectors.
Briefly, the current features that I can use to extract resources in the major search engines are word-based where each resource can be retrieved based on a series of words. (Note that this is not completely accurate, as there are a few other features that could be searched on such as language, file format, date of update, and domain suffix but the vast majority of the entries in the vector currently represent words).

Some Ideas (aimed at the search engines):

  • How about giving me a few more features to search on? For example, search engines seem to be storing away bolded and highlighted words to use in their prioritization schemes. How about exposing some of them so I can use them in a basic (or more advanced) search?
  • More advanced, give me the access to a large number of features that you are not already calculating, but should be (maybe you are already?). For example, I am always searching for interesting new technology product companies. Most of them have a tab-based link on their home page that says “product” (sometimes plural, but the stem is fine). How about a calculated feature that I can search on? This is one of a huge number of possible features.
  • Even better, let me make my own features calculations with some tools that you provide and you execute them and store them on your systems!
  • Just as important as the exposure to features, I would like to have the ability to make calculations off of the features and store them in your system. This will allow me to create some machine learning models (at a higher, concept level) and make the calculations in advance of my searches. (I will reduce the load on the systems by only uses specific features through this method, as I will use my composite variable for the resource extraction). This will also give me the ability to store some very advanced searches AND some of my schemes for prioritizing results. If you are really generous, you would allow me to make the calculations through the API as well.

I think Alexa already allows me to do this with its open platform, but I have not studied it enough at this point.

The net result is that I should be able to do some really interesting things with the information, especially if the other components of the open platform are in place (prior posts). The Amazon Camera Image search is one good example of what is possible (even if this does not interest you from a user standpoint, the search ability is amazingly specific).

Search Tech Opportunities- Improvement #5: Improved Tagging

Posted in innovate this!, Innovation at 3:29 pm by scottmaxwell

Following up my post on the opening of the search tech chain, the fifth opportunity that I see is with improved tagging. (While tagging may not be thought of in a traditional search sense, the reason for tags is to find things, so I include it here.)

The basic issue is that, while I love the tagging sites, I suck at tagging. When I find a site that I like, I would like to remember it, so I tag it and put it on Delicious or Wink, right? Well, it doesn’t work very well for me for several reasons. First, I have a hard time thinking about the tags that I should have. Second, I have a hard time remembering my tags. Third, the thought pattern that I have at the time of tagging is usually different than the thought pattern that I have at the time of retrieval. Fourth, the semantic meaning of one person’s tags can be very different than another person’s tags. Finally, those tag clouds definitely were not build with me in mind (they are attractive in an artistic way, but hard to use).

The net net of it is that a given resource ends up being tagged more generally rather than specifically, gets tagged based on a thought at the time of tagging, and gets tagged in a subset of all possible tags. This causes all sorts of retrieval problems.

Some thoughts on improvements to tagging:

  • Bare minimum, when I find a resource that I want to tag, let me know what others have tagged it and let me tick off the tags that I want to use for it.
  • Allow users to put together tag trees (already starting to happen. Wordrpess, for example, allows me to nest categories to two levels) that allow the tagger and the reader to better understand how a given tag fits into the world (as a side note, everyone knows that single taxonomy trees suck, but facited tree taxonomies are in my view the ultimate approach which is effectively this point. In my view, there can be lots of different overlapping trees that change over time, which allows for the “messy” world to be described better.)
  • Allow users to create tag trees that can be used by other users.
  • Use machine learning to propose possible trees and tags at the time of tagging of a resource AND to propose trees and tags to the user at the time of search/retrieval.
  • I am not sure the answer to the tag clouds, especially if others like them. perhaps a more organized way of representing how tags or trees relate would be helpful?

Again, there is an enormous amount of innovation potential here…

Next page