How Intelligent Is Your Community?

This post is a companion piece to a webinar I hosted with Carrie Melissa Jones, “Building Intelligent Communities: Applying the Principles of Intelligent Swarming to Scale Your Community Work.” In it, I’ll share ideas about measuring how intelligently your community is operating. As with the webinar, my thoughts here will be steeped in the research and writings of The Consortium for Service Innovation^TM .

Assessing the performance of a multi-layered system like a Community requires a multi-layered approach. The Consortium recommends Triangulation, which means comparing at least three types of data points to determine value, or success, or, in our case, Community Intelligence. I recommend looking at multiple data points from three different perspectives:

Community Members (a.k.a., the people your Community serves)
Employees (a.k.a., the people who work at your company and contribute to the Community)
The Business (a.k.a., your company)

Represented within these data points should be two types of indicators: Leading indicators–activities, or simply things you can count–and Lagging indicators, or outcomes/goals.

Here’s how you might chart it out:

	Community Members	Employees	Business
Leading Indicators	# of views of KB articles, documentation, blogs, white papers, webinars, etc. # of discussion forum posts that have answers and/or engagement Age and # of posts that remain unanswered # of answered posts that have been marked as “the best answer” # of answered post shares to answer duplicate questions # of kudos given to people who opt-in to answer questions % of community sessions in which people try to self-serve before contacting support	# of times employees create or update content when needed # of times employees use content to solve member issues # of times employees document the use of content that resolved issues # of times employee-created or updated content is used by other employees or members	# of known issues documented and available for self-service over time % of demand for customer support being satisfied within the Community $ saved when customer support demand that is satisfied within the Community would have otherwise landed in the Support org
Lagging Indicators	Self-service success Member speed-to-answer Member enthusiasm, engagement, and gratification	Employee confidence Employee enthusiasm, engagement, and gratification Employee understanding of the value they’re providing	Customer satisfaction Customer & employee effort Customer & employee retention Customer & employee learning Customer trust
	True community – people working together toward the same goals!

Don’t feel like you have to measure all of these things! To begin, pick 1-2 leading indicators and 1-2 lagging indicators from each stakeholder group that you can measure today. You can always add more as you gain access to additional insights or tools.

Finally, now that you have examples of what you can measure, you might be wondering HOW you’d measure some or all of this stuff. And the answer is, again, you’ll have to take a multi-layered approach.

Since leading indicators are activities, or at least all countable, they should be captured somewhere, like in your Community platform or search vendor analytics, your CRM, your CMS, your technical support ticketing system, or in various places from which you can pull the numbers and put them into a calculation.
Lagging indicators will be a combination of countable data points, surveys, user testing, and the observations of employees who live and breathe Community, like moderators and managers.

Finally, you should absolutely get a baseline for whichever measures you choose, or else you won’t be able to discern improvement! If you can’t get a baseline, you can always consider industry standards. The Consortium for Service Innovation and Technology & Services Industry Association (TSIA) are two of my main go-tos for industry standards and best practices.

Interested in learning more? Download the webinar for an overview of Intelligent Swarming^SM and lots of tips and tricks to help you build an Intelligent Community.