Answering the question: “Are we maturing?”
We’ve done a lot of thinking about how a Community of Practice (CoP) matures and have written a variety of posts on tools we use to grow healthy, sustainable communities. This post looks at how you might use data to prove that your CoP is maturing.
After a year since the launch of the KBW community, we wanted to know if the good vibes of the community were all in our minds or indeed backed up by data. We looked at how community members engaged with each other and the community of practice as a whole. See the full output in slide form.
We are analysing this data to have a look at how the community is progressing through growth, activity, and sense of community. Are we maturing? Let’s find out!
For a more complete walkthrough of the data we’ve collected, here is a video (transcript included!).
https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.loom.com%2Fembed%2F080e5da9106243ba92871be40b5e8e2c&display_name=Loom&url=https%3A%2F%2Fwww.loom.com%2Fshare%2F080e5da9106243ba92871be40b5e8e2c&image=https%3A%2F%2Fcdn.loom.com%2Fsessions%2Fthumbnails%2F080e5da9106243ba92871be40b5e8e2c-00001.gif&key=a19fcc184b9711e1b4764040d3dc5c07&type=text%2Fhtml&schema=loomA quick walkthough of how to collect data
Why you want to look at data
It’s awesome that you might feel that your community is healthy, but likely the community ambitions extend beyond being a friendly and inclusive place. As a community of practice evolves, you likely have specific goals, outcomes and activities you’re promoting.
To judge whether or not you are achieving these goals, you will want to look at how the community changes over time. We do this with data! Start by setting some baselines to measure against. With a data-driven approach, we can more accurately observe how a community is having impact.
Methodology and types of metrics
You’ll want to set a couple of different kinds of metrics, both qualitative and quantitative. You’ll want to look at how people engaged, how people were recognised and gather insights from the community members themselves. Your first step is to think about what you want to know.
1. Individual Engagement
Individual engagement is a way to look at how community members are interacting with one another and the community more broadly. Typically, you will want to look at things like:
- Did people introduce themselves?
- Did people made a contribution to or start a discussion?
- Did they join an event, either online or in real life?
- Did they sign up to courses or seminars on offer?
- What other signup or attendance rates can you collect?
Depending on the size of your community and the data capabilities of the platform(s) you’re using, you may need to come up with a sampling technique. Sampling allows you to select a fair and impartial subset of your community members to set engagement baselines.
The sampling technique we used for the KBW community was to select every 5th member who joined the community (listed by date). This technique allowed us to look at approximately 20% of the entire community, a decent sized sample.
2. Community Metrics
Many platforms have data capabilities of some kind. You might be able to look at:
- the total number of posts per month
- the total number of active members
- the total number of new discussions.
Setting baselines for posting activities can help you determine if the community is becoming more engaged.
The KBW community issues a variety of badges to recognise engagement and to encourage pro-social behaviours. With our community metrics, we were able to see how many badges were earned and what the most popular badges were. We found that 41% of the community earned at least one badge! Community members can earn badges that are discussed and displayed in the community as well as stealth badges. There are badges related to specific activities as well, such as Badge Wiki badges for people who contribute to the community’s knowledge base.
Looking at these metrics over the course of a year can help you determine ebbs and flows of activity. They can also help you see what kinds of content led to the most engagement.
3. Surveys
The third thing you should look at is what community members think and feel. Create an end of year survey focused on the key factors you’d like to know about.
- How do you want community members to feel?
- What kind of ambitions for the community do you hope people have?
- How do community members want to improve the community’s offerings?
For the KBW Birthday Survey, we used a Likert Scale and a subset of questions from McMillan and Chavis’s 1986 paper Psychological Sense of Community. We wanted to know if people felt like they belonged, if they identified with the values of the community and if their needs were being met. We also wanted to see if they thought the community had influence and trust.
The best part of surveying actual community members is the things they write in, so be sure and provide open text fields. People offer kudos as well as constructive criticism, so you’re sure to learn a lot about your community.
What’s next
Once you’ve collected and analysed your data, share it with your community! Other members of the community will have their own insights and ideas to make your community of practice even better.
We recommend putting on some music and diving into your community data. It’s easy to get excited about the things you learn because your data-driven insights can propel you, collectively, forward towards your goals.
Join us in the KBW community and learn more about communities of practice and all things badges