Philanthropists and foundations commonly have multiple investments, but are challenged to capture or summarize the collective impact of their resources and efforts, and evaluate whether they are truly progressing toward their ultimate goals. With the expansion of grant-making and the complexity of grant portfolios, EPAR offers a tool for reviewing investments that focuses on:
- Synthesizing: distilling and coding commonalities across investments to
- Describe, report on, and be able to communicate about the portfolio as a whole
- Identify synergies, patterns, redundancies, or gaps across the portfolio and network of investments
- Analyzing: the performance of
- Investments by outcome and impact, including social returns and public goods not easily measured in markets
- M&E systems for ongoing performance evaluation and learning
Over the past five years, EPAR has completed reviews of eleven philanthropic organization investment portfolios. Each portfolio review addresses a specific question, such as “How do our agricultural investments relate to nutritional outcomes”, or “What is, and how are we measuring, the impact of our investments on agricultural productivity?” Our review approach involves a review of the literature to develop a theoretical framework, which guides the review and coding of investment documents and the analysis of the portfolio. The manual review of program documents allows us to highlight trends and gaps in the grant documentation that relate to the specific questions of interest. The figure below illustrates our general portfolio review process.
Many steps in the portfolio process are similar to those for conducting literature reviews, including developing a theoretical review framework and using this to prepare a coding spreadsheet to organize the information from the review. The main difference is that the set of documents to review in a portfolio review is usually predetermined, and the unit of analysis is usually an individual grant or investment rather than a study, and the coding for each grant or investment may draw from many documents addressing different areas of interest for the review.
To facilitate certain aspects of the portfolio review process, EPAR has developed a suite of open-source text analytic tools in R and Python for reviewing large numbers of documents. The tools are used to identify patterns across a set of documents, and can provide a guide for human review. They provide a flexible framework for describing and classifying the content of textual documents, including analysis of word frequencies, description of common words, testing for correlations between words, and categorization of strings of text into modeled or human coded topic categories. Using machine tools to automate and focus parts of our manual analysis reduces costs associated with labor and time, and minimizes human error. A presentation describing the potential contributions of the EPAR text analysis tools in the context of an investment portfolio review process is available here.