We describe the “evaluability bias”: the tendency to weight the importance of an attribute in proportion to its ease of evaluation. We propose that the evaluability bias influences decision making in the context of charitable giving: people tend to have a strong preference for charities with low overhead ratios (lower administrative expenses) but not for charities with high cost-effectiveness (greater number of saved lives per dollar), because the former attribute is easier to evaluate than the latter. In line with this hypothesis, we report the results of four studies showing that, when presented with a single charity, people are willing to donate more to a charity with low overhead ratio, regardless of cost-effectiveness. However, when people are presented with two charities simultaneously—thereby enabling comparative evaluation—they base their donation behavior on cost-effectiveness (Study 1). This suggests that people primarily value cost-effectiveness but manifest the evaluability bias in cases where they find it difficult to evaluate. However, people seem also to value a low overhead ratio for its own sake (Study 2). The evaluability bias effect applies to charities of different domains (Study 3). We also show that overhead ratio is easier to evaluate when its presentation format is a ratio, suggesting an inherent reference point that allows meaningful interpretation (Study 4).