What is meant by a ‘healthy’ open-source community, and how do we capture it? These are the questions we’ve been asking within the Epiverse TRACE community. Built by an inclusive, global community of contributors, Epiverse TRACE comprises a suite of trustworthy, open-source, epidemiological software tools accessible to users across sectors. The intent is to enable the development of rapid, robust, and reproducible policy-relevant modeling.
In June 2022, the Epiverse TRACE community established blueprints for software development, considering not only technical elements (such as coding conventions) but also crucially, human interactions and the experience of participating in an open-source community. Incentivizing Collaborative and Open Research (ICOR)—a nonprofit initiative that seeks to facilitate and reward open science and collaboration—is working with us on tracking and measuring our goals toward this vision.
Metrics will become targets: make sure they mean something
An inspiration for our discussion were the metrics and analytics being developed for open-source community health by our colleagues at Community Health Analytics in Open Source Software (CHAOSS). They believe that metrics selected for one’s community will inevitably become targets, which means that the values underpinning the chosen metrics are critical. This was the impetus to begin formally defining the values we wished to inform the metrics for gauging the initiative’s collective success.
The values underpinning a community’s chosen metrics are critical.
Anna Carnegie Epiverse Community Manager London School of Hygiene & Tropical Medicine
Securing collective buy-in of core values
The selection of Epiverse TRACE core values involved the following stepwise actions, which took place over four months:
- Team audit of project documentation and discussions to date, plus a review of CHAOSS focus areas as a potential library of metrics to employ
- Draft documentation of values derived from the audit, indicating for each the purpose and potential metrics and data collection measures
- Circulation of draft document to the full Epiverse TRACE staff: 40 individuals with diverse backgrounds across four academic institutions and our funding partner, data.org
- In-depth interrogation of potential values, taking place over several weeks, with additional values proposed
For example, an exploration of the concepts of usability versus utility led to the determination that both were distinctive and crucial - Clarification and selection session, attended virtually by Epiverse TRACE staff across all partner sites, narrowed core values to 13 possibilities with the aim of selecting 7 to 8
- Final vote by team members resulting in a list of 8 core values selected and adopted
Epiverse TRACE community core values
The following list of values will represent the Epiverse-TRACE community, with metrics built out accordingly:
- Inclusion – ensuring that our community spaces are inclusive and welcoming to all.
- Sustainability of the ecosystem beyond the project’s grand-funded lifetime.
- Reciprocity – operating from a stance of give and take.
- Practicality of Epiverse TRACE tools and materials for end users – both in terms of usefulness and usability.
- Quality within the code development process.
- Timeliness of the tools developed (e.g., on a regular basis).
- Accountability to our community, our funders and each other.
- Environmentally conscious – taking steps as a project to minimise the negative consequences of our actions on the climate.
Next steps: metrics, communication, and ongoing assessment
We must now define appropriate metrics for capturing these values, taking care to incorporate both ‘hard’ and ‘soft’ measures. For example, when considering inclusion, in addition to recording the identity of those in attendance at virtual events and seminars, taking into account those who do or don’t feel empowered to speak.
Each of the core values will be embedded in our discussions, communications, and documentation. Embodying them will require accountability – i.e., admitting errors and making unpopular or difficult decisions. However, we anticipate that our systematic community approach will ameliorate these difficulties.
And as our community grows, there will be periodic reviews to capture new insights and wider perspectives, as well as to question the continuing relevance of each value. We see this as an ongoing conversation, one which we hope will be as engaging as the discussions to date.
To get involved with the community, please complete this short form.
Guest Author
Anna Carnegie is currently the Community Manager for the Epiverse initiative at LSHTM and MRC Unit The Gambia, which aims to fundamentally change how analytics are used in the global infectious disease response, moving towards scalable, community-driven software.
Read more