Although this book is mainly about how to launch and run new open source projects, that topic is inextricably linked to the problem of evaluating existing open source projects. You can't know whether you need to start a new project until you've evaluated what's out there (as explained in the section called “But First, Look Around”). Furthermore, even in a new project, you'll usually still be building on existing open source components, and will often be in the position of choosing between different projects that implement the same basic functionality. That is not just a technical choice; it's also about social health and general level of project maturity. How large and diverse are their developer communities? Do they get new contributors on a regular basis? Do they handle incoming bug reports in a reasonable way? Do they make stable releases frequently enough for your needs?
Evaluating open source projects is an art, not a science. However, there are some shortcuts that experienced people use. Below is what has worked for me. By "worked", I mean that when I have applied these evaluation techniques to a project and then checked in with that project months or years later, I have generally found its current status to be in line with what the evaluation predicted.
The most reliable reflections of project health can usually be found in the bug tracker. Look at the rate of issue filings and the number of unique filers (because that's a proxy for the size and level of engagement of the user base). Look also at how often project developers respond in bug tickets, and at how they respond: are they constructive? Do they interact well with both the reporter and with other developers? Is it always the same developer responding, or is responsiveness well-distributed throughout the development team? Are they inviting technically promising reporters to try becoming contributors?
More bug reports is better, by the way (as discussed in the section called “Version Control and Bug Tracker Access”). The rate at which bug reports are closed is not as important as you might think; in a healthy project with an active user base, bug reports are often filed faster than the development team can close them, especially when the user base is growing. The relevant signal is not the rate of resolution, but how project developers respond to and organize the influx of reports.
Look at the distribution of commits across committers, not just at the raw frequency of commits. Does the project have a variety of people working together in a sustained way? Too often, evaluators look just at the commit rate, but that rate isn't very informative — knowing the number of commits per week could just tell you that someone keeps making typos and then correcting them in new commits. If you have time to look at the content of individual commits, then look at how often one developer's commit is a response to (i.e., refers to) some other developer's previous commit. This tells you that group code review is going on, and the more of that you see, the better the project is doing.
In addition to looking for a variety of individual identities, see if you can tell how many different organizations are participating in the project — in particular, commercial organizations. If a number of different sources of money are all investing in a project, that's a sign that that project is going to be around for the long term. (See also the discussion of "bus factor" in Chapter 4, Social and Political Infrastructure.)
If the project has discussion forums, scan them quickly looking for signs of a functional community. Specifically, whenever you see a long thread, spot check responses from core developers coming late in the thread. Are they summarizing constructively, or taking steps to bring the thread to a decision while remaining polite? If you see a lot of flame wars going on, that can be a sign that energy is going into argument instead of into development.
Any project that is functioning well will usually have made announcements within the past few months. Check the project's front page, news feed, Twitter or other microblog accounts, etc. If things are quiet on stage, they're probably quiet backstage too.
This is just a brief introduction to the art of evaluating projects, but even using just the steps above can save you a lot of trouble. I have found them particularly useful when evaluating the two sides of a recent fork.[84] Even in a recent fork, it is often possible to tell, just by looking at some of the signals described above, which side will flourish over the long term.