Product evaluations available on the Digital Impact Exchange are intended to guide users on the maturity of the tool, as ranked by the indicators shown. These indicators are compiled from assessments conducted by different organizations (such as Digital Square) and by using automated metrics from GitHub and code repositories of the software. These maturity scores are not an endorsement but a reflection of ratings given by various organizations.

Updated Indicators (to be read from GitHub):

Explanatory text

Actively maintained tools indicates that the product maintainer is able to manage the velocity of work required to keep the product up to date. We are considering several indicators for this overall category:

1. The first indicator is a ratio of open to merged pull requests. A pull request is a mechanism for developers to notify team members that they have completed a feature. Once the pull request has been reviewed, the product maintainer can merge the feature and close the pull request. Closing issues is a measurement that reflects the ability of the product team to close incidents and provide service to the product’s community.

This score is calculated reading the information of open pull/merged requests from GitHub using the following range:

- N/A = 0 (*Note, this could indicate that the software is not using GitHub)

- Low (Merged Pull Requests:Open Pull requests, ratio is less than 1)

- Medium (Merged Pull Requests: Open Pull requests, ratio is between 1 and 5)

- High (Merged Pull Requests: Open Pull Requests, ratio is greater than 5)

2. The second indicator is a ratio of open to closed issues. Closing issues is a measurement that reflects the ability of the product team to close incidents and provide service to the product’s community.

This score is calculated reading the information of open/closed issues from GitHub using the following ranges:

- N/A = 0 (*Note, this could indicate that the software is not using GitHub)

- Low (Closed issues: Open issues, ratio is less than 1)

- Medium (Closed issues: Open issues, ratio is between 1 and 2)

- High (Closed issues: Open issues, ratio is greater than 2)

3. The third indicator is looking at the date of the last repository activity to determine how current the updates are. With open source software, it can be difficult to assess if the tool is current and has an active community and product maintainer behind it or if it has sunsetted, and this indicator can help shed light on this.

This is score is calculated by reading information from GitHub and by using the following ranges:

- Low (Last Repository Activity more than 365 days)

- Medium (Last Repository Activity between 30-364 days)

- High (Last Repository Activity less than 30 days)

Explanatory text:

Software quality is an important measure, but it is difficult to quantify. By assessing how others are interacting with the software, we can make assessments of its usage, as a proxy for quality. We use several indicators (read from GitHub) for this category:

1. Software releases are an indication of the stability of the system; frequent software releases enable a system to recover and respond to changes in an agile manner. 

- N/A = 0 releases (*Note, this could indicate that the software is not using GitHub for tracking their releases)

- Low (Releases = 1)

- Medium (Releases 2-9)

- High (Releases = 10 or more)

2. Downloads - downloads are a good proxy to indicate usage of a tool and demonstrates that a project is successfully converting an audience and attracting the right audience.

- Low = less than 9 downloads

- Medium = 10-99 downloads

- High = 100+ downloads

3. Stars - GitHub stars can help show how many people have noticed the project, as an indication for its popularity, its usefulness, or as a product of the product team’s efforts in promotion and communications.

- Low = 99 or less stars

- Medium = 100-999 stars

- High = 1000+ stars

4. Forks - a project fork happens when developers start independent development on a software package, creating a distinct and separate piece of software. In the case of development forks, there are positive reasons for forking, such as to revive a discontinued software or to meet varying user needs. However, forking can also imply a separation of the developer community.

- Low = 49 or less forks

- Medium = 50-99 forks

- High = 100+ forks

5. Watchers - watchers are Github users who have asked to be notified of activity in a repository, but have not become collaborators. It demonstrates their interest in the project.

- Low = 49 or less watchers

- Medium = 50-99 watchers

- High = 100+ watchers

Explanatory text

A software that can attract contributors means the tool being supported by a healthy community.

The score is calculated by reading number of contributors and commits from GitHub and by using the following ranges:

1. Contributors - number of contributors indicates that they perceive the tool as a way to help them improve their coding skills, gain experience, increase community and peer recognition, for greater job prospects, and to improve a tool that they use themselves.

- Low (2 or less)

- Medium (2-10)

- High (more than 10)

2. The number of “commits” indicates how often a developer is revising their code and engaging with the software. Committing often allows developers to share their work more frequently with others to avoid having to merge conflicts.

- Low = Commits 49 or less

- Medium = Commits 50-999

- High = Commits 1000+

 

Explanatory text

Code review helps developers learn the code base, as well as help them learn new technologies and techniques that grow their skill sets. This score is calculated by with a number of indicators: 1) does the software use a top 25 language? 2) Is it containerized, 3) Does it have a readme, 4) Does it have a valid license file, 5) does it have API documentation?

1. Top 25 language - Using a common programming language means that there will be a large pool of developers, tools, and support, ensuring that the project will be able to be maintained into the future.

The scores are calculated using the following ranges:

- Low (Language less than 29)

- Medium (Language = 11-25)

- High (Language greater than 25)

2. Containerization allows developers to write applications once and run them everywhere. That level of portability is essential in terms of developing process and vendor compatibility. It also has other benefits, for example, fault isolation, security and ease of management.

This score is calculated as a binary:

Yes = 1

No = 0

3. Since ReadMe files are usually the first item users interact with in a repository, it is important that ReadMe files provide users with the information needed to engage with the corresponding repository.

This score is calculated as a binary:

Yes = 1

No = 0

4. An up-to-date software license and copyright helps determine the legal maturity of a product. Without an open source license, the software component is unusable by others, even if it has been publicly posted on GitHub. Each open source license states what users are permitted to do with the software components, their obligations, and what they cannot do as per the terms and conditions.

This score is calculated as a binary:

Yes = 1

No = 0

5. An open API is a valuable type of technology for developers. It provides standards for how software programs communicate with each other. With an open API, developers have to write less code to make different software programs work together seamlessly.

This score is calculated as a binary:

Yes = 1

No = 0

 

 

Digital Square Indicators (only for those products that have been vetted by Dig Sq.)

This product has been assessed by Digital Square. Global utility assesses how widely the tool is used, how well funded it is, and other metrics measuring its adoption and potential for use. 

This product has been assessed by Digital Square. Community Support assesses both support for a community of users (such as documentation and multilingual support) and engagement of the user community with the tool.

This product has been assessed by Digital Square. Impact assesses the level of development of the software in such areas as security, interoperability, technical documentation, and scalability.